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ABSTRACT 


This thesis performs a comparative analysis of a sampling of risk management 
plans for strategic and critical suppliers administered by the Defense Contract 
Management District West (DCMDW) in order to identify the areas of highest risk and 
the most common tools used to mitigate risk in key processes and systems for these 
suppliers. 

The Defense Contract Management Agency (DCMA) uses a comprehensive, 
inclusive, and iterative approach to risk management. It follows the Government and 
DoD risk management premise of using a five-step approach to risk management and the 
basic idea of identifying and assessing key processes/systems whose risk, either through 
probability or potential impact, offers the most cause for concern firom a performance, 
schedule, or cost perspective. It employs current information technology. Risk 
Assessment and Management Program (RAMP) to provide consistency, commonality, 
access, and comparability to its risk management process. 

Performance and schedule, product support and supplier quality assurance for 
product quality, and delivery were the areas of highest risk for DCMA. The most 
commonly applied risk handling tools indicated in the RAMP database were areas 
associated with analysis, monitoring, and surveillance activities before final inspection: 
“Data Analysis”, “Product Audits”, “System Evaluation”, and “Corrective Action”. 


V 



THIS PAGE INTENTIONALLY LEFT BLANK 


VI 



TABLE OF CONTENTS 


I. INTRODUCTION-1 

A. PURPOSE-1 

B. BACKGROUND-2 

C. RESEARCH OBJECTIVE- 3 

D. RESEARCH QUESTIONS_4 

1. Primary Research Question---.....-4 

2. Subsidiary Research Questions...................................................-4 

E. SCOPE---4 

F. METHODOLOGY_5 

G. ORGANIZATION OF STUDY-6 

II. RISK AND RISK MANAGEMENT BACKGROUND-7 

A. RISK AND RISK MANAGEMENT DEFINED-7 

1. RiskCharacteristics.............................................................................7 

2. Risk Management Characteristics .....................................................9 

B. RISK MANAGEMENT IN THE FEDERAL ACQUISITION 

PROCESS_11 

1. The Risk Management Process......................................—..............12 

2. Post-award Administration Phase.............................—...................16 

C. DCMA RISK MANAGEMENT PROGRAM-18 

1. PhUosophy...........................................................................................l9 

2. Process Oriented Contracting Administration Services 

(PROCAS)_19 

3. Integrated Product Teams (IPTs)...........—...........................—......20 

4. Management Councils .......................................................................21 

D. CHAPTER SUMMARY_.22 

HI. RISK MANAGEMENT PLANS-25 

A. INTRODUCTION.-.25 

B. SUPPLIER RISK MANAGEMENT_26 

1. Risk Planning.....................................................................................26 

2. Risk Assessment---...----—......27 

3. Risk Handling.....................................—..........................................30 

4. Risk Monitoring....................—......—............................................31 

5. Risk Documentation-.....--—--31 

C. RISK ASSESSMENT AND MANAGEMENT PROGRAM (RAMP).....32 

1. Transition...........................................................................................32 

2. Organization......................................................................................34 

3. Responsibilities-....---.35 

4. Risk Rating Assignments..................................................................35 

5. Supplier Risk Handling .....................................................................36 

6. Government Monitoring ..................................................................37 

vii 










































7. Risk Management................._.........._........_...._..._.....38 

D. CHAPTER SUMMARY_39 

IV. RAMP DATA PRESENTATION AND ANALYSIS_41 

A. INTRODUCTION_41 

B. OVERALL RISK RATINGS & SERVICE SET SUMMARY_42 

1. Performance...._..............___.......___.....43 

2. Schedule..............................................................................................43 

3. Cost......................................................................................................43 

C. MAJOR PROGRAM RISK RATING_44 

1. Earned Value Management......._ ...45 

2. Acquisition Logistics Support...........................................................48 

D. PRODUCT SUPPORT RISK RATING_50 

1. SPRD&E - Design Engineering.._____...._....._..52 

2. SPRD&E - Systems Engineering .....................................................55 

3. Test and Evaluation_........_......._......___...._ ....57 

4. Configuration and Technical Data Management............................59 

5. Parts Management Program .....___......___........60 

6. Software CAS......................_............................_......._.....62 

7. Supplier Quality Assurance — Quality System ................................65 

8. Supplier Quality Assurance - Product Quality_68 

9. Packaging Management Program............._.........._...._......_73 

E. DELIVERY RISK RATING_75 

1. Schedule and Delivery Management................................................76 

2. Contract Safety Requirements..........................................................81 

F. BUSINESS AND FINANCIAL SYSTEMS RISK RATING_82 

1. Contractor Estimating System Reviews...........................................84 

2. Material Management and Accounting Systems_ 86 

3. Contract Property Management ......................................................87 

4. Contractor Purchasing System Reviews_....___90 

G. PAYMENT AND FINANCIAL MANAGEMENT RISK RATING_92 

1. Progress Payments Based on Cost.._......_.....___....93 

2. Public Vouchers .................................................................................94 

3. Performance Based Payments_..............................................96 

H. VARIABILITY IN THE SAMPLED PLANS_98 

1. Missing Risk Ratings .........................................................................98 

2. Key Process/System Choice_ 99 

3. Risk Handling Tools ........................................................................100 

I. CHAPTER SUMMARY_101 

V. CONCLUSIONS AND RECOMMENDATIONS_103 

A. OVERVIEW_103 

B. CONCLUSIONS_103 

1. Subsidiary Research Question 1: What is Risk Management in 
the Context of the Federal Acquisition Process? ..........................103 

2. Subsidiary Research Question 2: What is the Defense 


Contract Management Agency (DCMA) Philosophy with 

viii 














































regard to Risk Management in the Post-award Contract 
Administration Phase? ....................................................................104 

3. Subsidiary Research Question 3: Are Risk Management Plans 

for Specific Activities Consistently Developed and Applied 
within DCMA?_106 

4. Subsidiary Research Question 4: What are the Areas of 
Highest Risk for Strategic and Critical Suppliers in the 
Contract Administration Phase? ....................................................107 

5. Subsidiary Research Question 5: What are the Most Common 
Tools used to Mitigate Risk in Key Processes and Systems? .......109 

6. Primary Research Question: How Does the Defense Contract 
Management Agency (DCMA) Address Risk Management in 
the Acquisition Process?..................................................................110 

C. RECOMMENDATIONS_111 

1. DoD Should Mandate a Common Risk Management Process 
throughout all DoD Organizations and Applicable to Each of 
the Services, Agencies, and Acquisition Offices ............................111 

2. Revise the RAMP Plan Format to.Make them Even More 
Directly Comparable to Each Other and Incorporate a 
Summated Spreadsheet Linked to the Risk Ratings for Each 
Area.............................................—...........................—.................112 

3. Use RAMP Data for Past Performance Information-113 

D. SUGGESTED AREAS FOR FURTHER RESEARCH-114 

1. Identify and Compare Various Risk Management Models and 

IT Systems in Use in DoD---114 

2. Study the RAMP Program from an IT and Process Oriented 
Perspective......................................................................... .•••••—••••••114 

3. Research Whether RAMP and Other Risk Management 

Activities at DCMA Actually Reduce Acquisition Risk-115 

APPENDIX A. RAMP INVENTORY-117 

APPENDIX B. RISK RATING OVERVIEW-119 

LIST OF REFERENCES_123 

INITIAL DISTRIBUTION LIST-125 


IX 



















THIS PAGE INTENTIONALLY LEFT BLANK 


X 


LIST OF FIGURES 


Figure 2.1. DSMC Risk Management Structure (From RM, 1999).13 

Figure 2.2. FAP Post-Award Administration Phase (After Ross, 1999).17 

Figure 3.1. Risk Analysis Process (From SRM Brief, 2001).29 

Figure 3.2. Supplier Risk Management (After OB, 3.1,2001).32 


XI 






TfflS PAGE INTENTIONALLY LEFT BLANK 


Xll 


LIST OF TABLES 


Table 3.1. Supplier Risk Management Process (After OB, 3.1,2001).27 

Table 3.2. Service Set Alignment (After SRM Brief, 2000).34 

Table 4.1. Overall Risk Ratings.42 

Table 4.2. Major Program Service Set Risk Ratings.45 

Table 4.3. Key Processes/Systems for Earned Value Management.46 

Table 4.4. Key Processes/Systems for Acquisition Logistics Support.49 

Table 4.5. Overview of the Service Set Risk Ratings for Product Support.51 

Table 4.6. Key Processes/Systems for SPRD&E-Design Engineering. 53 

Table 4.7. Key Processes/Systems for (SPRD&E)-Systems Engineering.56 

Table 4.8. Key Processes/Systems for Test and Evaluation.58 

Table 4.9. Key Processes/Systems for Configuration and Technical Data 

Management.60 

Table 4.10. Key Processes/Systems for Parts Management Program.61 

Table 4.11. Key Processes/Systems for Software CAS.63 

Table 4.12. Key Processes/Systems for Supplier QA-Quality System Risk 

Management Efforts.67 

Table 4.13. Key Processes/Systems for Supplier QA-Product Quality.70 

Table 4.14. Key Processes/Systems for Packaging Management Program.74 

Table 4.15. Overview of the Service Set Risk Ratings for Delivery Risk.75 

Table 4.16. Key Processes/System for Schedule and Delivery Management.77 

Table 4.17. Key Processes/Systems for Contract Safety Requirements.81 

Table 4.18. Overview of the Service Set Risk Ratings for Business and Financial 

Systems Risk. 83 

Table 4.19. Key Processes/Systems for Contractor Estimating Systems Review.85 

Table 4.20. Key Processes/Systems for Material Management and Accounting System 

(MMAS).87 

Table 4.21. Key Processes/Systems Chosen for Contract Property Management.88 

Table 4.22. Key Processes/System for Contractor Purchasing System Review.;.91 

Table 4.23. Overview of the Service Set Risk Ratings for Payment and Financial 

Management Risk.93 

Table 4.24. Key Processes/Systems for Progress Payments.94 

Table 4.25. Key Processes/Systems for Public Vouchers.95 

Table 4.26. Key Processes/Systems for Performance Based Payments.97 


Kill 































THIS PAGE INTENTIONALLY LEFT BLANK 


XIV 



ACKNOWLEDGMENTS 


First, I wish to thank those without whom my thesis would not be complete: CDR 
Jim Barnard, Dr. Shu Liao, and CDR Dave Smith. 1 would especially like to recognize 
Ms Keah Shields whose tireless efforts and unwavering energy made my thesis possible. 

Second, I would like to say my time here has been made all the more special 
because of those with whom I have shared it. Hats off to “The Group”: Pixie, Miss E, 
Millie, MyT, Mikey, Petey, Tri-Mike, Suzy-Q, and Josh ... you are the best. I will 
always remember you; I will never forget this place or the times we have shared. You 
have made my tour here unforgettable and unmatched. 

Third, I wish to thank Michelle, my wingman, my buddy, my pal, my confidante 
extraordinaire\ Fair winds and following seas to you and your family. 


XV 




THIS PAGE INTENTIONALLY LEFT BLANK 



I. INTRODUCTION 


A. PURPOSE 

Risk is the probability of an undesirable event and the significance of its 
consequence, or more succinctly, “an event and its probability and impact (RM, 1999). 
Within the development of major projects, there are five common facets of risk: 
technical, programmatic, supportability, cost, and schedule and they can be found in all 
phases of the Federal Acquisition Process from procurement planning and requirements 
analysis to the award and post-award phases. The Defense Systems Management College 
(DSMC) defines the risk management structure for Department of Defense (DoD) 
acquisition as a continuous, iterative activity between key processes, described as risk 
planning, risk assessment, risk analysis, and risk handling. The Defense Contract 
Management Agency (DCMA) is intricately involved in the transition of major 
acquisition programs from award to performance, and specifically in the post-award 
contract administration phase of risk management. To handle these responsibilities, 
DCMA utilizes risk management plans designed to incorporate all aspects of a successful 
risk management program. 

The purpose of this thesis is to perform a comparative analysis of a sampling of 
risk management plans for strategic and critical suppliers administered by the Defense 
Contract Management District West (DCMDW) in order to identify the areas of highest 
risk and the most common tools used to mitigate risk in key processes and systems for 
these suppliers. Through a comprehensive literature review, a sampling of actual risk 
management plans, and information gathered through interviews with DCMA personnel 
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within varying DCMA resident and regional offices in DCMDW, a study of common 
high risk areas and risk mitigation techniques is developed to aid DCMA personnel in 
developing future risk management plans and techniques. 

B. BACKGROUND 

All acquisition programs are subject to risks. The DCMA One Book defines risk 
as a measure of the inability to achieve overall program objectives as defined by cost, 
schedule and techmcal goals. In this context risk is generally described by its probability 
of occurrence and its impact. To guard against risk, DCMA has established risk 
management as an operating principle and an integral part of its processes. 

Risk management is a systematic approach to problem solving. It includes risk 
planning, assessing risk areas, developing risk handling options, monitoring risks to 
determine how they have changed, and documenting the overall risk management 
program. Risk management plans are plans of action to reduce or e liminate; risks 
affecting cost, schedule or performance. By identifying, analyzing and managing risks 
through an iterative, continuous program assessment via risk management plans, DCMA 
can have significant positive impacts on the cost, schedule, and performance of its 
assigned programs. 

Risk management plans identify and track key risk drivers, define risk abatement 
plans and provide for continuous risk assessment. Through the use of risk management 
plans specific to each of its suppliers, DCMA seeks to identify and control critical risk 
functions and bring them within acceptable levels. The Risk Assessment and 
Management Program (RAMP) is an information technology tool that readily allows 
DCMA to define and document risk management plans and to share the information 
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contained in these plans throughout the organization, and ultimately to its external 
customers. 

In RAMP, DCMA categorizes its suppliers as “strategic”, “critical”, or “routine”. 
Strategic and critical suppliers represent those contractors or contract actions of highest 
significance (from a cost or safety standpoint) and therefore highest risk—with respect to 
potential impact: 

• Strategic suppliers are ACATI prime contractors. 

• Critical suppliers are those not designated as “strategic” and who 

produce products/services classified as: ACAT I sub-contractor 

(delegation), safety of flight, flight critical, life support, explosives, 
munitions, hazardous, specialized safety, level 1 sub-safe, nondestructive 
test, de mili tarization, engaged in First Article Testing at the time of risk 
assessment (when RAMP capable), space/satellite (when RAMP capable), 
or nuclear (when RAMP capable). 

• Routine suppliers are all those not designated as “strategic” or “critical”. 
(Shields, 2001) 

By analyzing the plans for strategic and critical suppliers, we can readily assess 
how consistently DCMA is applying its own philosophies, identify the more significant 
or high risk areas of commonality among suppliers, and recognize the more prevalent risk 
handling tools used to mitigate against risk in the post-award contract administration 
phase of acquisition. 

C. RESEARCH OBJECTIVE 

The objective of this thesis is to identify and examine risk in the post-award 
contract administration phase of the Federal Acquisition Process. The goal of this study 
is to identify commonalities in the various risk management plans of strategic and critical 
suppliers, determine the areas of highest risk and define common tools used to mitigate 
risk in key processes and systems for these suppliers. 
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This research serves as a case study of the risk management process in DCMDW. 
The research will benefit DCMA offices in their continued implementation of the RAMP 
program and its integration into their current risk management progr ams and processes. 
The ability to see beyond the immediate supplier and recognize the commonalities that 
exist in managing risk in the post-award contract administration phase of Government 
acquisition enhances the ability of DCMA personnel to make sound business assessments 
and implement reasoned risk management approaches that have a proven track record, a 
higher probability of success, and are consistent with Government and DoD risk hanHIing 
guidance. 

D. RESEARCH QUESTIONS 

1. Primary Research Question 

How does the Defense Contract Management Agency (DCMA) address risk 
management in the acquisition process? 

2. Subsidiary Research Questions 

• What is the Defense Contract Management Agency (DCMA) philosophy 
with regard to risk management in the post-award contract administration 
phase? 

• Are risk management plans for specific activities consistently developed 
and applied within DCMA? 

• What are the areas of highest risk for strategic and critical suppliers in the 
contract administration phase? 

• What are the most common tools used to mitigate risk in key processes 
and systems? 

• What is risk management in the context of the Federal Acquisition 
Process? 

E. SCOPE 

This thesis will be a case study. The effort will be directed to analyzing risk 
management plans as documented in the RAMP database available to DCMDW. 
Interviews and opinions of key Government representatives involved in RAMP 
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implementation and the risk management program will augment the study. This research 
will not provide an exact template for risk management plans rather it will provide an 
analysis of various risk management plans for strategic and critical suppliers assigned to 
the DCMDW and seek to identify commonalities that exist between them to draw 
conclusions regarding areas of high risk and risk handling tools in the post-award 
contract administration phase of federal acquisition. 

Specifically, this thesis will (1) review risk in Federal acquisition and specifically 
in the post-award contract administration phase; (2) present the current DCMA Risk 
Management program, processes, and systems; (3) analyze a representative sampling of 
DCMA risk management plans for various strategic and critical suppliers in DCMDW; 
(4) identify commonalities in the development and application of Risk Management 
Plans; (5) discuss areas of highest risk in the contract administration phase; and (6) 
identify common tools used to mitigate risk in key processes and systems. 

F. METHODOLOGY 

This thesis is a study of risk management plans for strategic and critical suppliers 

assigned to DCMDW. It includes identification of commonalities in risk management 

plans, an assessment of high risk areas common to strategic and critical suppliers, and a 

presentation of common tools used to mitigate risk in key processes and systems. A 

comprehensive literature review of books, magazine articles, CD-ROM systems, Internet 

based materials. Government reports, corporate materials and other information sources 

is conducted to describe risk in the acquisition environment and the risk management 

background within which the DCMA offices operate. The Defense Systems Management 

College (DSMC) Risk Management Guide for DoD Acquisition and the DCMA Supplier 

Risk One Book Chapter are used as guides for identifying risk areas and risk handling 
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treatments. A sampling of various risk management plans for strategic and critical 
suppliers is obtained using the RAMP database through the DCMDW office. These plans 
are analyzed to identify commonalities in their development and application and 
specifically, to look for high-risk areas and common risk handling tools used to mitigate 
risk for strategic and critical suppliers. 

G. ORGANIZATION OF STUDY 

Following this introductory chapter. Chapter II provides background information 
on risk management in the Federal Acquisition Process. It also provides an overview of 
the DCMA Risk Management Program including its philosophy and various aspects of 
the overall program to include Process Oriented Contracting Administration Services 
(PROCAS), Integrated Product Teams (IPTs), and Management Councils. Chapter III 
examines risk management plans in the DCMA environment. The Risk Assessment and 
Management Program (RAMP) is reviewed including the development and application of 
risk management plans throughout the agency. Chapter IV presents data obtained from a 
sampling of risk management plans for strategic and critical suppliers in DCMDW and 
analyzes the commonalities found within their plans, areas of high risk, and risk handling 
tools. Chapter V includes the conclusions and recommendation of the thesis. It answers 
the research questions and addresses topics for additional research. 
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n. RISK AND RISK MANAGEMENT BACKGROUND 


A. RISK AND RISK MANAGEMENT DEFINED 

Webster’s defines risk as the “possibility of loss or injury”. (MW, 2001) Other 

generic definitions include “chance of something going wrong” (Encarta, 2001) or, when 

used as a verb, “accept the danger of doing it”. (Cambridge, 2001) This seems simple 

enough, but when asked “what is risk?” the term becomes hard to grapple with and, as 

with a lot of things, the best answer may be “it depends”. Risk is a dependent concept 

and typically one that is thought of in negative terminology. One needs to know the 

context with which it is being used relative to time and space to properly define it so that 

it actually means something. 

In acquisition related terms, DoD defines risk as 

... a measure of the potential inability to achieve overall program 
objectives within defined cost, schedule, and technical constraints ... [it] 
has two components; (1) the probability (or likelihood) of failing to 
achieve a particular outcome, and (2) the consequences (or impact) of 
failing to achieve that outcome. (RM, 1999) 

1. Risk Characteristics 

Risk can be characterized in certain emotive and descriptive words. Basic 
characteristics of risk include volatility, variance, imcertainty, ignorance, incomplete 
knowledge and ambiguity. Each of these terms defines various aspects or dimensions of 
risk dependent upon the perspective fi'om which it is viewed. (Shapira, 1995) Over time 
and in common use, risk has evolved fi’om an unintended or unexpected outcome to an 
outcome and a chance of its occurrence that is decidedly unfavorable. (Ansell, 1992) 
Risk, therefore, is commonly thought of in a negative context. 
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Yet other methods for describing risk seem much more scientific—^unemotive in 
nature. The George Washington University’s Educational Services Institute (ESI) course 
on risk management characterizes risk as situational, time based, interdependent, 
magnitude dependent, and value based (ESI, 1998). These variations objectively describe 
and distinguish the nature and identity of risk in the subjective context of its environment. 

In the context of decision-making, risk choices seem to embrace three dominate 
aspects: risk definition, risk attitude, and risk management. How is risk defined in the 
situation at hand? What are the decision-makers’ attitudes—risk adverse, risk neutral or 
risk seeking? How will the risk be dealt with? (Shapira, 1995) 

a. The Definition of Risk 

Risk can be thought of or described in terms of its dimensions and its 
relation to imcertainty. Outcomes, both positive and negative can be considered. 
Parameters can be established to consider how much risk is used and whether 
combinations of risk will be regarded as descriptive of the whole. (Shapira, 1995) 

b. Attitude Toward Risk 

The relationship between risk and return will mold the approach towards 
seeking or avoiding risk in varied situations dependent upon people, resources, situations, 
etc.. The degree of risk and its level of consequence will likewise impact the prevailing 
responses. (Shapira, 1995) 

c. Dealing with Risk 

The methods used to handle or treat risk will be dependent upon whether 
risks are to be avoided, delayed, or reduced. Decisions include whether to attempt to 
control risk, gather more information, or merely change the parameters or estimates. 
(Shapira, 1995) 
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2. Risk Management Characteristics 

Risk management is “the act or practice of dealing with risk. It includes planning 
for risk, assessing (identifying and analyzing) risk areas, developing risk handling 
options, monitoring risks to determine how risks have changed, and documenting the 
overall risk management program.” (RM, 1999) 

The risks at issue may be actual, but they are also definitely what is perceived. 
Decisions are made based on perceptions of risk consequences. Perceptions are estimates 
of probabilities or likelihood and evaluations of magnitude of outcomes that are often 
subjective; they are psychologically derived. Further, there is a political dimension 
whereby decision makers are influenced by those affected by the outcomes. So despite 
the classical decision theoiy premise of objective calculations, risk management involves 
many subjective and judgmental contributions. (Ansell, 1992) 

Given this, successful risk management requires (1) flexible and general models, 
(2) a family of related methods which link models to circumstances, (3) a wide range of 
skill and expertise, and (4) experience and leadership. (Ansell, 1992) Risky choice is 
succinctly conflict resolution. Managers are expected to manage, not just assess and 
accept risk; they are expected to “make things happen” and “take (good) risks”. (Shapira, 
1995) 

Taking a less esoteric look and once again seeking to more scientifically describe 
risk management, the ESI comse identifies four major components of risk management: 
risk identification, risk quantification, risk response development, and risk response 
control. (ESI, 1998) These concepts are developed further below. 
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a. Risk Identification 

Risks should be identified early, often, regularly, and at all levels— a. 
comprehensive approach. The process should be thorough and fully documented. Tasks 
should be assigned to specific team members. Inputs can include such items as 
requirements document, work breakdown structure, cost and time estimates, etc. Tools 
for idea generation include expert interviews and brainstorming, etc. (ESI, 1998) 

b. Risk Quantification 

Quantifying risk includes analysis and prioritization Analysis includes 
worst, best, and most probable scenarios. Assess probabilities and determine impacts 
such as schedule risk, cost risk, profitability, etc. (Quantifiable measures are preferred, 
but qualitative can be used.) Rank analyzed risks, highest to lowest and filter out 
unimportant risks. (ESI, 1998) 

c. Risk Response Development 

Plan and implement basic risk response strategies based on risk type. 
Evaluate and select a primary option based on a strategy of acceptance (of the 
consequences), avoidance (eliminate the cause), or mitigation (minimiz e probability, 
minimize impact or transfer the risk). (ESI, 1998) 

d. Risk Response Control 

Risk response control involves implementing risk strategy, evaluating and 
documenting the results. As risks become actual events, strategies are carried out. 
Clearly define the lines of responsibility, communicate status, and dociunent actions. 
Evaluate the results to reassess risk probability, impact and events as well as risk 
strategies. Assess risk as to cost, schedule and performance. Continually document risk 
results: current, accurate, complete, and simple. (ESI, 1998) 
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B. RISK MANAGEMENT IN THE FEDERAL ACQUISITION PROCESS 

Acquisition reform has changed the field of play where the Department of 

Defense must buy its wares. Technical, business and management approaches have 
evolved. Today, it’s all about commercial products, streamlined processes, and best 
value. This dynamic arena is juxtaposed against a backdrop of trimmed defense budgets 
and reduced Government oversight that makes for a risky playing field. 

Risks can be thought of in terms of future events and the uncertainty associated 
with their occurrence and potential impact. They are inherently interdependent, time- 
based and obviously situational. In acquisition related terms, this means risks exist 
throughout the life of a program and at all phases of the acquisition cycle. A risk 
occurrence in one area or phase will absolutely effect risk elsewhere, e.g., a slip in 
schedule early on in the process will have domino effects downstream and potentially 
impact other risk areas such as cost. 

The Federal Acquisition Process (FAP) segregates acquisition into three distinct 
phases: pre-solicitation, solicitation-award, and post-award administration. Risk is alive 
and well in each of these areas. The pre-solicitation phase includes risk associated with 
such fimctions as needs determination, market research, requirements analysis and 
sourcing. The award phase problems include issues such as solicitation method, selection 
of contract type, bid or proposal evaluation, and award selection. Transitioning firom 
award to contract administration involves risk associated with contract administration 
plans, early DCMA involvement (early PROCAS), post-award orientation conferences 
(the handoff) and flow-down clauses for subcontractors. (Ross, 1999) 
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Risk management activities span across all phases and ftmctions of the acquisition 
cycle. Area emphasis, scope and detail will vary according to phase and depend upon the 
specific risk event. Though there is no one standard prescribed for use, there are some 
general requirements and basic processes that are common throughout the DoD 
acquisition arena. 

1. The Risk Management Process 

The Department of Defense mandates the use of risk management in its major 
defense acquisition programs: 

The acquisition strategy shall address risk management. The PM 
[Program Manager] shall identify the risk areas of the program and 
integrate risk management within overall program management.” 

Further, DoD encourages the use of risk management throughout the 
entire program life cycle and advocates “life cycle risk management 
versus risk avoidance. (DoDD 5000.2-R, 2001) 

Given that risks are to be managed vice avoided, DoD describes the overarching 
risk management process: 


The establishment of a risk management process (including planning, 
assessment (identification and analysis), handling, and monitoring) to be 
integrated and continuously applied throughout the program, including, 
but not limited to, the design process. The risk management effort shall 
address risk plaiming, the identification and analysis of potential somces 
of risks including but not limited to cost, performance, and schedule risks 
based on the technology being used and its related design, manufactur ing 
capabilities, potential industry sources, and test and support processes; risk 
handling strategies; and risk monitoring approaches.... (DoDD 5000.2-R, 

2001) 

The Defense Systems Management College (DSMC) reiterates this basic risk 
management process and details its structure and make-up. This then forms the basic 
process model DoD prescribes to deal with acquisition related risk. 
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Figure 2.1. DSMC Risk Management Structure (From RM, 1999). 

a. Risk Planning 

Risk planning is the process to develop a risk management strategy; 
determine the methods used to identify, analyze, handle, monitor, and document risk; and 
plan for adequate resources to implement the program. The result is a Risk Management 
Plan (RMP) that is iterative and descriptive of the schedules, activities, and processes. 
The plan is in essence, a road map. (RM, 1999) 

b. Risk Assessment 

Risk assessment is the identification and analysis of risk. The process 
begins with the compilation of risk events and the subsequent evaluation at a level of 
detail to understand causality i.e. risk drivers and impact. This problem identification is 
the stage that quantifies the probability and consequences of various risks. 

Risk assessments t 5 q)ically include a performance/technical assessment, a 
schedule assessment, and a cost estimate. Risk analysis activity begins with a detailed 
study of the critical risks to judge the probability and impact on cost, schedule, and 
performance. Risk ratings are assigned and are often expressed as high, moderate, and 
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low based on consideration of the likelihood of the risk event’s occurrence and its 
consequences. (RM, 1999) 

However, it is worth noting here that there is no one mandated method for 
assessing or classifying risk within DoD. For example, there is no requirement to classify 
risk as “high”, “moderate”, and “low” and no specific method for prioritizing the risks 
following the initial assessment. Given this predicament, it becomes difficult to compare 
risk or how risk is handled between various programs and activities or between the 
services themselves. 

c. Risk Handling 

Risk Handling is the specific methods and techniques used to deal with the 
identified risk. The chosen options are a direct result of the risk assessment ra ting and 
prioritization. It includes scheduling, the assignment of responsibility, and provides cost 
estimates. The objective is to manage risks to acceptable levels. Risk handling options 
can include risk avoidance, control, transfer and assumption. (RM, 1999) 

(1) Risk control seeks to mitigate risks to reduce the likelihood 
and/or consequence of their occurrence. It includes such activities as trade Studies, early 
prototyping, incremental development, modeling/simulation, reviews/inspections, and 
manufacturing screening. (RM, 1999) 

(2) Risk avoidance seeks to eliminate high or medium risk 
sources and replace them with a lower risk solution. This process may involve such 
efforts as changes in the requirements or specifications. An up-front requirements 
analysis and cost-as-an-independent variable (CAIV) trades are sample risk handling 
options here. (RM, 1999) 
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(3) Risk assumption acknowledges a risk situation and 
consciously accepts the associated risk level without especially seeking to exert any 
control over it. The basic premise here is that not all risks are worth worrying about. To 
handle risk in this maimer resources (time, money, people, administration) must be 
identified to overcome the risk should it occur. (RM, 1999) 

(4) Risk Transfer reallocates risk to another part of the system 
during the design phase or re-distributes risk between the Government and prime 
contractor or between Government agencies or contracting team members. It is a form of 
risk sharing and can influence cost objectives depending on where the burden is placed. 
(RM, 1999) 

d. Risk Monitoring 

Risk monitoring systematically tracks and evaluates the risk handling 
activities against established metrics. It is a reiterative process which can likely result in 
changing and identifying new risks and risk handling methods. The key measures here 
are cost, schedule and performance effects. This is basically a feedback technique. Test 
and Evaluation (T&E), demonstration events, program metrics, and process proofing are 
sample risk monitoring techniques. (RM, 1999) 

e. Risk Documentation 

Formal documentation of the risk management process offers several 
benefits. It serves as a basis for assessments and updates, ensures a more comprehensive 
assessment, provides a method for monitoring and verifying results, provides background 
material, is useful as a management tool, and produces rationale for program decisions. 
(RM, 1999) 
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2. Post-award Administration Phase 

The Post-award Administration Phase of Federal acquisition (also known as 
Contract Administration) includes among other functions; start-up, quality assurance, 
payment and accoimting, contract modification, claims, termination and closeout. These 
can be further delineated as described in Figure 1.2. The first four functions are required 
for every acquisition, while the last three are dependent upon the administrative 
requirements specific to the contract in question. Within this myriad of functions are 
many risks and many associated methods of handling them. 

The Government and contractor plan and initiate performance in the post-award 
phase. Large dollar contracts or contracts for complex, technical requirements require a 
contract administration plan to delineate Government surveillance and monitoring 
activities and provide for proper Government and contractor performance. 

Various means and methodologies are used to perform these administrative 
functions. Agencies may assign a Contracting Officer’s Representative (COR) or 
Contracting Officer’s Technical Representative (COTR) to liaison between the 
Government and contractor to provide technical assistance to the contractor and current 
contract information to the Contracting Officer (CO). The Defense Contract 
Management Agency (DCMA), the principal organization for handling contract 
administration within DoD acquisition, assigns Administrative Contracting Officers 
(ACOs) to perform specifically delineated contract administration functions. (Ross, 
1999) 
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POST-AWARD ADMINISTRATION PHASE 


FUNCTIONS 

SUB-FUNCTIONS 

TASKS 

Start-Up 

Planning 

Ordering 

Subcontracting 

Contract Administration 
Planning 

Post-Award Orientations 

Order Against Contracts 

Consent to Subcontract 

Quality Assurance 

Monitoring and Problem 
Solving 

Property 

Reporting Performance 
Problems 

Monitor, Inspect, and 

Accept 

Delays 

Stop Work 

Remedies 

Property Administration 

Report Performance 

Problems 

Payment and 
Accoimting 

Payment 

Accoimting 

Limitation of Costs 

Payment 

Unallowable Costs 

Assignment of Claims 

Collecting Contractor Debts 

Progress Payments 

Price and Fee Adjustments 

Accounting and Cost 
Estimating Systems 

Cost Accounting Standards 

Defective Pricing 

Closeout 

Closeout 

Closeout 

Contract Modification 

Contract Modification 

Contract Modification 

Termination 

Termination 

Termination 


Bonds 

Claims 

Claims 

Claims 


Figure 2.2. FAP Post-Award Administration Phase (After Ross, 1999). 

Post-award contract risk management naturally follows all that has gone before; it 
builds on what is already in place. The process starts with an Integrated Baseline Review 
(IBR) following contract award to ensure plans and performance baselines are adequate 
and consistent with the contract schedule, scope, and resources. Although specific steps 
to initiate the risk management plan will vary, the following identifies some of the more 
basic ideas: 
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• Conduct initial meeting with contractor to describe the objectives and 
approach to risk management. 

• Train Government contract administrators and contractors’ organization 
on risk management basics. 

• Review pre-contract award risk plan and revise as necessary. 

• Conduct in-depth review of risk assessments and expand as necessary. 

• Review and revise risk handling plans to match adjustments made in the 
assessment. 

• Review documentation requirements with the contractor and Government 
administration staff. 

• Establish a formal risk management organization consistent with contract 
terms. 

• Refine risk monitoring plans with the contractor. 

• Establish program reporting requirements with the contractor. 

• Identify other risk management activities in conjimction with the 
contractor. 

• Manage the program risk in accordance with the risk management plan. 

• Work with contractor to refine risk monitoring plans and procedures; 
develop performance measures and metrics to track medium and high risk 
items. (RM, 1999) 

C. DCMA RISK MANAGEMENT PROGRAM 

DCMA is DoD’s contract manager responsible for ensuring programs, supplies, 

and services are delivered on time, within cost, and meet performance requirements. As 
of January 26, 2001, the DCMA homepage reported DCMA manages 325,000 prime 
contracts valued at $852 billion and employs over 12,000 civilian and military 
professionals. Their employees interact with customers on a daily basis to ensure 
customer needs are met: 

• Before Award - assist in designing solicitations, identify potential 
performance risks, select capable contactors, and write contracts 
promoting easy contract administration 

• After Award - ensures contractor product, cost, and schedules are in 
compliance with the terms and conditions of the contract to include on-site 
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surveillance and program-specific processes that cannot be performed by 
off-site buying activities (DCMA web site, 2001) 

DCMA’s mission includes providing risk assessment services. DCMA manages 
risk using integrated supplier surveillance planning. They preposition workforce at 
contractor sites and use integrated management teams to bring a multifunctional 
perspective to assess supplier systems and processes for cost, schedule, and 
product/service performance. (DCMA web site, 2001) 

1. Philosophy 

The DCMA One Book (OB) establishes risk management as a central operating 
principle and integral part of DCMA processes. DCMA follows the standard risk 
management philosophy prescribed in the DoD Risk Management Guide (RM) and the 
Defense Acquisition Deskbook (DAD). (OB 0.1, 2001) 

DCMA’s risk management methodology complies with statutory and regulatory 
requirements and is specific to the conduct of contract administration processes. Agency 
risk management is comprised of five steps: risk planning, risk assessment, risk 
handling, risk monitoring, and risk documentation. (OB 0.1,2001) Five mechamsms are 
used to carry out the risk management process: Integrated Product Teams (IPTs); 
contractor documentation, product/processes, metrics and data; Process Oriented Contract 
Administration Services (PROCAS); inspection/audit results and data analysis; and 
process mechanisms for individual assessment tools. (OB 0.1,2001) 

2. Process Oriented Contracting Administration Services (PROCAS) 

PROCAS promotes mutual trust and understanding between the Government and 

contractors by using common objective data to improve performance and encourage 

successful contract completion. PROCAS adds value by increasing customer satisfaction 

through improved contract performance from improved processes and increased on-time 
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delivery. It is a continuous process improvement approach that systematically provides a 
method for selecting, analyzing, and modifying processes so that once a problematic 
process has been identified and corrected, the IPX then moves on to the next process. 
(OB 0.3,2001) 

All DCMA employees must use PROCAS methods to the maYitrmm extent 
possible. Process improvement efforts must be conducted using Integrated Product 
Teams (IPTs) and must be prioritized according to highest returns to the Government. 
DCMA offices must maintain a history of PROCAS efforts including identification of 
key processes, process analysis, risk classifications, resource adjustments and associated 
cost savings or cost avoidance. (OB 0.3,2001) 

DCMA Contract Management Offices (CMOs) must determine the degree of 
involvement based on cost, schedule, and technical risks in the selected processes 
affecting delivery or service. They must develop surveillance plans to form the basis for 
risk assessment and process identification and prioritization. (OB 0.3, 2001) However, 
contractors are responsible for the processes employed to fulfill Government contracts 
and it is their choice whether to choose to team with the Government to improve 
processes of mutual interest. DCMA members can never mandate process 
improvements. (OB 0.3,2001) 

3. Integrated Product Teams (IPTs) 

IPTs are characterized as multi-functional and multi-organizational teams 
designed to take advantage of the disparate skills of their members. IPTs add value by 
bringing various functional disciplines together to jointly build programs. Customers and 
Management Councils select areas of potential benefit fi-om IPT involvement to resolve 
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problems, improve performance, facilitate reform initiatives, and develop surveillance 
plans. IPTs must use PROCAS techniques and risk assessment tools to improve overall 
contract performance. (OB 0.3,2001) 

IPTs are designed to promote cooperation and full and open discussions. Team 
members should be qualified (in their fimctional disciplines) and empowered to speak for 
their superiors or “principals” in the decision-making process. In fact, they are 
encouraged to frequently communicate with their leadership to ensiare they are espousing 
sound advice to the sponsoring party (Management Coxmcils, customers, etc.) and should 
make other team members aware of any of their limitations to speak for their parent 
organization. Agreements are considered “final” and therefore this continuous “up-the- 
line” comm uni cation is essential. (OB, 0.3,2001) 

4. Management Councils 

Management Coimcils are senior representatives from customer buying activities, 
program management offices, the Defense Contract Audit Agency (DCAA), DCMA and 
contractors. They are forums to communicate ideas, implement change and speed up 
improvements in acquisition. Here, all the stakeholders are brought together to 
coordinate and resolve issue and thereby add value to the acquisition process. (OB 0.3, 
2001) 

CMOS must establish and support councils at all contractor sites that have major 
acquisition programs (ACAT I and II), hold greater than 80% of their unliquidated 
obligations, or on an as needed basis. Coimcils cannot alter, amend or deviate from 
contract terms and FAR/DFARS requirements. (OB 0.3,2001) 
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Contractor members must have the authority to represent their corporation across 
at least two business areas, contractual entities or profit centers. All members must be 
senior enough to commit resources and make decisions for their organizations. Central to 
this is the decision and the power to establish IPTs and target specific processes for 
increased risk management. The CMO Commander, the DCAA Resident Auditor and all 
Program Managers and Item Managers must be council members. (OB 0.3,2001) 

D. CHAPTER SUMMARY 

This chapter defines risk, provides some distinguishing characters and then 
describes it in the context of decision-making. Risk management is then introduced 
conceptually as a way to deal with risk. It is characterized by some basic over-arching 
functions that run a common thread through federal acquisition and DoD risk 
management. Finally, the DCMA risk management program is discussed and some of its 
more salient components presented. 

In decision-making, risk choices can be characterized in three dimensions: the 
definition of risk, the attitude toward risk, and dealing with risk. Risk management is the 
process used to deal with risk and includes the four broad steps of identification, 
quantification, response development, and response control. 

The Federal Acquisition Process divides procurement into three phases: pre¬ 
solicitation, solicitation-award, and post-award administration. Varying risks and risk 
treatments exist throughout the cycle, but the basic DoD risk management process 
remains consistent and is mandated for use in major defense acquisition programs: risk 
planning, risk assessment, risk handling, risk monitoring, and risk documentation. It is an 
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iterative process and is carried through, continued, and expanded upon in the post-award 
contract adminis tration phase of the acquisition process. 

DCMA is the principal contract administrator for DoD. Risk assessment services 
are central to their mission of taking care of customer needs. DCMA follows the 
standard DoD risk management process and uses several mechanisms to carry out this 
function. IPTs, PROCAS, and the use of Management Councils are central to the risk 
management philosophy employed to ensure the contractor provides customer service 
and product delivery. 

The next chapter will look at risk management plans and how they are used and 
incorporated into DCMA’s new risk management database, the Risk Assessment and 
Management Program (RAMP). RAMP is designed to incorporate all aspects of the risk 
management process and provide a common tool whereby information can be easily and 
quickly shared with geographically dispersed administrative offices and customers. 
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III. RISK MANAGEMENT PLANS 


A. INTRODUCTION 

DCMA has adopted a comprehensive risk management methodology that is to be 
applied consistently to all its suppliers. Their Supplier Risk Management Program 
integrates the assessment and monitoring processes and is consistent with the stated DoD 
five-step process of risk planning, assessment, handling, monitoring, and documentation. 
DCMA has recently employed a new tool, the Risk Assessment & Management Program 
(RAMP), a computer software application, to assist them in accomplishing their risk 
management mission. 

RAMP facilitates an integrative and iterative approach to the risk management 
process. It provides DCMA with several valuable functions in carrying out its contract 
administration and subsequent risk management mission: 

• Provides one standard automated tool to assess cost, schedule, and 
performance risk. 

• Facilitates collection and documentation of supplier risk information. 

• Requires supplier involvement prior to input. 

• Shares information with buying activities (customers) in the form of an 
integrated risk management plan. 

In keeping with its efforts to promote teaming and cooperation with contractors as 
encouraged with the PROCAS, IPX, and Management Council initiatives, DCMA 
principles dictate that suppliers should be informed and involved with risk management 
actions and results. DCMA further stipulates that the generated output of the RAMP 
system is not to be used by buying activities as past performance data and its use as 
source selection information is also limited. (OB, 3.1,2001) 
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B. SUPPLIER RISK MANAGEMENT 

The DCMA “One Book” chapter on Supplier Risk Management establishes the 

risk management policy and methodology DCMA CMOs use to document risk 

statements and the required monitoring levels and techniques they use in response to 

specific contractor facility risk. All personnel use the planning, assessment, handling, 

monitoring and documentation approach to perform these efforts at supplier facilities. 

Through risk management, DCMA determines the priority, degree, and intensity of risk 

handling and monitoring as well as required resources needed at specific CMO locations. 

(OB, 3.1, 2001) A detail of the DCMA risk management process and responsibilities 

follows. 

Table 3.1. presents an overview of the various sub-processes within the risk 
management process as a whole. 

1. Risk Planning 

DCMA Manages risk in the post-award contract administration phase through the 
use of CMOs assigned to all of its suppliers. All suppliers and/or contractual agreements 
must have associated risk management plans. Functionally integrated CMO teams 
review the contract and customer requirements to gain a clear xmderstanding of the 
customer needs and expectations. Through this procedure CMOs identify key processes 
and technical and business systems that will require surveillance. Key processes are 
identified by their “consequence of failure” on contract performance, schedule, or cost. 
(OB, 3.1) 
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SUPPLIER RISK MANAGEMENT PROCESS OVERVIEW 


Process Inputs: 

Sub processes: 

Contract, Purchase Order & Modifications 

Risk Planning 

Memorandum of Agreement (MOA), Quality 

Risk Assessment 

Assurance Letter of Instruction (QALI), 

Risk Handling 

Letter of Delegation (LOD) 

Risk Monitoring 

FAR&DFARS 

Performance Based Assessment Model 

Risk Documentation 

(PB AM) Users Guide 

Process Mechanisms: 

Performance Based Business Environment 

Fxmctional personnel & IPTs 

(PBBE) Guides 

Contractor documentation, products/ 

Supplier policies, procedures, standards. 

processes, metrics, and data 

and data 

PROCAS 

Work Breakdown Structure (WBS) 

Supplier and program office risk 

Inspection/audit results & data analysis 

Management plans 

Process Controls: 

Formal/informal reviews 

Contractual terms & conditions 

Customer feedback 

Customer requirements 

Pre-award surveys 

CMO management review 

DLA-GC Notifications of Suspect Product 

Unit Self-Assessment (USA) 

Internal Operations Assessment (10A) 
Management Control Reviews (MCRs) 

DLAD 5000.4 Processes: 

Configuration Mgt including Technical Data 

Performance Based Payments 

Contractor Estimating System Reviews 

Progress Payments Based on Cost 

Contractor Purchasing System Reviews 

Property Control System Analysis 

Contract Safety 

Public Vouchers 

Earned Value Management System (EVMS) 

Schedule and Delivery Management 
Software Contract Admin Services 

Integrated Logistics Support 

(SWCAS) 

Material Mgt & Accounting Systems 

Supplier Quality Assurance (QA) 

Packaging Management Program 

System Planning, RD&E 

Parts Control Program 

Test and Evaluation Management 


Table 3.1. Supplier Risk Management Process (After OB, 3.1,2001). 

2. Risk Assessment 

DCMA must perform a risk assessment for all its suppliers. Performance, 

schedule, and cost are the principal areas of consideration. The CMO team or functional 

specialist will assign a risk rating to each system or key process based on a combination 
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of high, moderate, and low ratings for likelihood of failure and impact. The key 
indicators of risk are a contractor’s experience, performance, and capability. The rater 
must be able to support the assigned risk ratings with actual data representative of these 
key indicators: proofing, audits, evaluations, etc; both contractor and Government 
supplied information can be used for this purpose. For assistance, the rater can research 
each applicable One Book process and its associated Risk Matrix to ascertain specific 
performance requirements that relate to the contractor in question. (OB, 3.1,2001) 

Figure 3.1 depicts the risk assignment process using a matrix table to produce risk 

ratings. 


The following risk ratings are appropriate under the listed conditions. Key 
definitional differences are denoted by the italicized words. 
a. High Risk 

• Failure or nonconformance likely to result in unsafe conditions for 
personnel. 

• Failme of nonconformance likely to result in mission failure or prevent 
proper tactical function of a major end item (aircraft, weapon, or space 
system) 

• Process is out of control. 

• Performance data indicates significant doubt of system or process 
capability to meet requirements. 

• A major disruption is highly probable and the contractor is unlikely to 
meet performance, schedule or cost objectives. 


28 





b. Moderate Risk 

• Failures could result in unsafe conditions. 

• Failures could adversely affect mission performance. 

• Proper performance of end items, subassemblies, or key processes is 
doubtful. 

• There is a moderate process variance and the trend is adverse. 

• Performance data indicates doubt of system or process capability to 
consistently meet requirements. 

• Probable that the contractor will encounter delays and if concerns are not 
addressed the process may progress to “high” risk. 

c. Low Risk 

• Failures are unlikely to present serious problems. 

• Performance data provides confidence in system or process capability to 
meet requirements. 

• Minimal or no impact in meeting performance, schedule, or cost 
objectives. (08,3.1,2001) 
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3. Risk Handling 

DCMA teams or specialists must use risk handling plans as the operational risk 
management tool. The plans must specify the methods used to mitigate risk associated 
with a contractor’s systems or key process. CMOs may use either Government action or 
Contractor Self-Oversight as the surveillance method. 

IPTs or functional specialists develop and execute risk handlin g plans as required 
according to applicable DLAD 5000.4 One Book policy chapters or as indicated by other 
technical and business systems presenting risk. The risk handling plan indicates the 
intensity, schedule and frequency of the designated risk handling method. CMO 
personnel can apply PROCAS methods (process proofing, product audits, data analysis, 
etc.) to any of the risk areas to improve systems, processes, or products. (OB, 3.1, 2001) 
Some examples of risk handling methodologies, given risk specific situations follow. 

a. High Risk 

• Immediate and intensive surveillance. 

• Establish intensive system evaluations, product audits, process proofing, 
data analysis, root cause analysis, corrective action, and statistical 
sampling. 

• Execute until risk is mitigated to a lower level. 

b. Moderate Risk 

• Intensity and frequency of surveillance includes establishment of 
scheduled system evalimtions, product audits, process proofing, data 
analysis, root cause analysis, corrective action, and statistical sampling. 

• Execute until risk of impact is reduced. 

c. Low Risk 

• Intensity and frequency of surveillance includes using periodic 
Government and contractor data reviews (EVMS, delivery performance 
history, process control data, cost control data, extensive audit data, etc.) 

• Ensure process variance does not increase and process capability remains 
stable. (OB, 3.1,2001) 
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CMOs must have a specific risk handling plan for all suppliers at all given 
locations regardless of complexity, risk level, or dollar value of contract(s). The plans 
must be tailored to the program, contract, or supplier facility. The depth and length of the 
plans vary and depend upon business volume, product criticality, or acquisition 
complexity. The plans may be contract specific (when the requirement is not applicable 
to all contracts within a facility) or facility specific (when the process or system is 
common to all contracts within the facility). (OB, 3.1,2001) 

4. Risk Monitoring 

The DCMA team or specialist must track and evaluate performance relating to 
systems and key processes identified in the risk handling plan. Monitoring involves 
constant and consistent follow-up of all that has gone before through the regular use of 
surveillance methods that will truly measure contract performance. Assigned personnel 
will compare results with objectives for the various risk handling methods and adjust the 
methods, intensity, and frequency accordingly. This is basically trend analysis, an 
important indicator of future success. Adverse results may require the IPT or specialist to 
take corrective measures and increase surveillance. They will modify risk assessments 
and the risk handlin g plans as needed to account for the results of the ongoing risk 
management program. (OB, 3.1,2001) 

5. Risk Documentation 

The team or specialist must record and maintain current documentation of the 
entire risk management program and any updates as required. (OB, 3.1,2001) 

Figure 3.2 displays the Supplier Risk Management Process as a flow chart, clearly 
indicating the iterative nature of the risk management process. 
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SUPPLIER RISK MANAGEMENT PROCESS FLOW 



Figure 3.2. Supplier Risk Management (After OB, 3.1,2001) 

C. RISK ASSESSMENT AND MANAGEMENT PROGRAM (RAMP) 

1. Transition 

RAMP software is the mandated tool for risk assessment and handling activities 


throughout DCMA. It integrates and automates these processes and eases collection and 


documentation of supplier risk information. RAMP is consistent with DCMA 
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Information Technology policy for mission applications and replaces all local automated 
risk assessment tools that were previously in use. RAMP is a module in the web-based 
Supplier Information Service (SIS) and is open to DCMA customers, basically supplying 
the same information previously shared through other channels: IPTs, Management 
Councils, etc. (IM 00-223,2000) 

By implementing RAMP, DCMA CMO personnel have transformed from 
conducting periodic risk assessments using a Performance Based Assessment Model 
(PBAM) that required only tri-aimual full-up evaluations, with annual desk audits 
dispersed in between, to real-time supplier siuweillance. The new process integrates the 
PBAM risk assessment and surveillance planning processes to institute a consistent risk 
management methodology throughout DCMA. (TM 99-79, 1999) When implementing 
RAMP, CMO personnel are able to initially populate the RAMP database with previously 
used PBAM information due to the fact that the tenets of the program remain consistent 
with the new policies. (IM 00-223,2000) 

A new Supplier Risk Management One Book Chapter was added to be the 
“overarching” policy for the new risk management program. Additionally, 20 One Book 
Chapters “link to” and supplement the guidance. Process Owners were required to 
update policy, guidebooks and training strategy to accommodate these changes. The Risk 
Assessment and Management Program (RAMP) is the automated tool associated with the 
supplier risk management program. (IM 99-273,1999) 

The RAMP database will remain closed to DCMA suppliers due to the fact that it 
is intended solely to be a DCMA internal management tool designed to automate policy. 
However, DCMA operating principles encourage a teaming approach with its suppliers. 
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as evidenced through other acquisition reform initiatives: PROCAS, IPTs, Management 
Coimcils, etc. With this in mind, supplier information is to be shared and discussed with 
the cognizant contractors prior to use within the RAMP system 

2. Organization 

A RAMP risk management plan is organized to assign risk ratings at four 
different levels: Overall, Service Set, One Book chapter, and key process/system. Five 
service sets support the overall rating and 20 One Book Chapters and their associated key 
processes/systems define the service set. CMO personnel will assign three separate 
ratings for performance, schedule, and cost to identified risks at each of these levels. If 
no risk is identified, then the area will remain un-rated. However, an “overall” rating 
must be assigned to the supplier or contract as a whole. (IM 00-293,2000) 


The following table displays the five service sets employed in the RAMP database 


and the associated 20 One Book Chapters used to evaluate contractor risk: 


ONE BOOK POLICY STRUCTURE 

SERVICE SET ALIGNMENT 

Major Program 

Earned Value Management System (EVMS) 
Acquisition Logistics Support 


Delivery 

Schedule and Delivery Management 
Contract Safety 


_ 


Business and Financial Systems 

Contract Property Management 

Contractor Estimating System Reviews 
Contractor Purchasing System Reviews 

Material Management & Accounting Systems 


Product Support 

System Planning RD&E - Design Eng 
SPRD&E - Systems Eng 

Test & Evaluation Management 

Supplier QA - Quality System 
Configuration & Teclmical Data Mgt 
Packaging Management Program 

Parts Control Program 

Software CAS 

Supplier QA - Product Quality 




Payment & Financial Mgt 

Progress Payments Based on Cost 

Performance Based Payments 

Public Vouchers 





_ 


Table 3.2. Service Set Alignment (After SRM Brief, 2000). 
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3. Responsibilities 

Functional specialists populate the initial database and enter data into RAMP. 
They rate their processes and systems and overall One Book Chapters in performance, 
schedule, and cost. This rating is a professional judgment call that should take into 
accoxmt supporting and verifiable information. This is justified by a written narrative that 
describes the information used to support the ratings. (IM 00-293,2000) 

Service Set ratings are system generated from their supporting One Book ratings 
and cannot be altered. CMO designated Supervisor/Team Leaders \vill review the ratings 
at this level and provide a written narrative that summarizes the assessment and 
prescribed risk handling activities. (IM 00-293,2000) 

The CMO designated Operations Group Leader or Team Leader(s) vdll review all 
RAMP information and assign an overall rating to performance, schedule, and cost and 
write a supporting narrative. These ratings are system generated, but can be changed at 
the discretion of CMO management. (IM 00-293,2000) 

4. Risk Rating Assignments 

A risk rating of “high”, “moderate”, or “low” is assigned to performance, 
schedule, and cost for the supplier or contract overall and for the identified risk areas at 
each of the other four levels. Some service sets or One Book chapters may receive no 
rating at all: “NA” or “not applicable”. 

If there are no historical contractor data, second party data, or working records for 
a new contractor, the key processes or systems identified for risk assessment should be 
considered “in process” for performance, schedule and cost until data can be reviewed. 
This is a temporary rating imtil a functional specialist can review first output. There 
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should however, be rationale (narrative) for this area since it was chosen as a priority for 
risk assessment in the first place. (IM 00-293,2000) 

In and of themselves, areas that buyers require DCMA to monitor are not 
automatically considered “high” risk. The risk rating assigned by the CMO is a 
combination of likelihood/probability and impact should the risk event occur. Customer 
specified “important” characteristics are a contributing factor when specialists rate the 
impact or consequence side of the risk matrix. (IM 00-293,2000) 

All delegated subcontract work shall be entered into RAMP to provide customer 
visibility of this level. If the contractor’s delegation only specifies product 
characteristics, DCMA personnel should identify the actual subcontractor process(es) that 
produce these specified product characteristics. (IM 00-293,2000) 

5. Supplier Risk Handling 

DCMA risk assessments and the resulting risk ratings are designed to be based on 
verifiable and producible data that contractors can review. Government judgment calls 
alone are usually not enough to convince a contractor to take additional measures to 
guard against potential error. The data used to support the risk rating may be 
Government or contractor collected, so long as they produce clear evidence. (IM 01-020, 
2001) Although the Government may plan and assess risk, it is the supplier who must 
actually handle it to change the way a process works. 

As part of its assignment in facilitating DoD’s risk management program, DCMA 
has a role to influence the risk handling the contractor may voluntarily perform. This is 
where the “narrative” aspect of RAMP assessment often proves to be helpful by 
providing a clear cause-and-effect trail for the contractor to consider. A “cooperative” 
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approach similar to a PROCAS agreement is the preferred course of action within 
DCMA. Failing this, a Corrective Action Request (CAR) is an alternative tool that may 
need to be utilized by the CMO. (IM 01-020,2001) 

Once risk is identified, suppliers may choose one of fovir risk handling options to 
deal with a risky process or system: 

• Avoid risk by changing the situation so that risk is no longer present i.e. 
restructuring. 

• Accept risk by acknowledging its likelihood and consequences and 
(hopefully) plan for its contingency if it occurs. 

• Transfer risk to another system or location where the impact is 
minimized. 

• Control risk by reducing the likelihood (prevention) or the impact 
(reduction). (IM 01-020,2001) 

6. Government Monitoring 

This is actually risk handling performed by the Government. Since the 
Government cannot actually alter the process or system—^having no such ownership over 
these areas—DCMA must conduct continuous data review to “pulse” identified key 
processes/systems considered risky to the contractor’s overall performance. This 
basically involves gauging the movement of measured outputs (or trend analysis) from 
the risk handlin g tools chosen to mitigate the risk. 

Intensity, frequency and schedule are used to describe the risk handling tools for 
each key process/system identified as requiring risk management. “Intensity” measures 
the degree to which the specific tool is to be applied, e.g., 100%, sample size, specific 
elements. (IM 00-293, 2000) “Frequency” describes the periodicity of the risk handling 
action and “schedule” provides a more specific time reference. 
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This area of the risk management process also includes any “mandatory 
inspections” customers require. From the DCMA perspective, an inspection is only as 
good as its ouqjut’s relevancy to managing risk: risk is not reduced unless a process 

is changed and inspection does not change a process.” (IM 01-020, 2001) Inspection is 
the monitoring of performance and assists in determining whether the risk handling 
methodology needs to be changed to improve performance. 

7. Risk Management 

RAMP produces what is more accurately considered to be Risk Management 
Plans because they encompass all five aspects of the risk management process (planning, 
assessment, handling, monitoring, and documentation) and not just the “risk handling” 
requirements called for under DCMA’s supplier risk management policy. (IM 01-020, 
2000) 

In maintaining its status as a real-time risk management tool, RAMP will be 
updated as needed to report current conditions at supplier locations. As prescribed by 
DCMA policy, the maximum frequency between updates is one year. Personnel 
responsible for updating the database (the functional specialists) are tasked with keeping 
abreast of changing conditions at contractor sites that could result in changing risk ratings 
and priorities, e.g., reorganizations, strikes, renovations. (IM 00-293,2000) 

RAMP is intended to facilitate the collection of supplier information for the 
purpose of contract management. It provides the framework for a systemic approach to 
assigning risk ratings that are used by DCMA personnel to identify and prioritize process 
improvements as well as resource allocation. However, despite this substantial gathering 
of performance information on specific contractors, DCMA has established policy that 
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expressly prohibits the use of RAMP data for pre-award source selection past 
performance information. “RAMP is intended to be used as a post-award system, not as 
a past performance tool because it does not have the appropriate checks and balances 
necessary for that purpose.” (IM 01-115,2001) 

D. CHAPTER SUMMARY 

This chapter begins with a discussion of DCMA’s supplier risk management 
process including the risk analysis matrix used for assigning risk ratings and the 
associated risk handling methodologies applied to the various rating levels. It proceeds 
with DCMA’s incorporation of current risk management information into the new 
automated risk management program, RAMP. Finally, various aspects of the RAMP 
database are reviewed and pertinent program application issues detailed. 

DCMA has adopted a comprehensive risk management methodology to 
consistently apply to all its suppliers. DCMA created a new One Book Chapter to 
describe its risk management process and assign responsibilities to its CMOs. It employs 
a risk matrix structure to define risk in terms of probability and consequence and assign 
risk ratings for performance, schedule, and cost. Risk handling methodologies vary in 
intensity as appropriate to mitigate the associated level of risk. 

The RAMP program is designed to be an all encompassing risk management plan 
incorporating all five fimctions of the risk management process in one automated tool 
allowing users fi:om geographically dispersed sites to share data. The RAMP database 
will be initially populated with existing risk management plans. Its information will be 
made available to customers although it is expressly not to be used for past performance 
data or source selection criteria. RAMP will remain closed to suppliers; however, in 
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keeping with the tenets of PROCAS, IPTs, and Management Councils, the information 
will be shared and discussed with contractors prior to use. 

RAMP is the mandated tool for risk assessment and handling throughout DCMA. 
DCMA functional specialists or IPTs will identify risk priorities and assign risk ratings at 
the key process/system and One Book Chapter level; management/supervisory oversight 
will review automatically generated ratings at the Service Set and Overall rating levels 
and write narrative cause-and-effect descriptions to support the assigned risk. However, 
the role of risk handling belongs to the contractor; it is the supplier’s process that must be 
adjusted and only the contractor can do this. Hence, the ongoing teaming aspects of 
DCMA’s risk management program. 

The next chapter will present risk management data from a sampling of risk 
management plans representative of the Defense Contract Management District West 
(DCMDW) region. A comparative analysis of these plans, obtained from the RAMP 
database, will be conducted to identify commonalities, high risk areas, and risk handling 
tools consistent across the region. 
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IV. RAMP DATA PRESENTATION AND ANALYSIS 


A. INTRODUCTION 

The pxirpose of this chapter is to present and analyze risk management data 
obtained from a sampling of risk management plans from the RAMP program initiated in 
DCMDW. Forty-two (42) RAMP plans from strategic and critical suppliers are 
reviewed. The analysis focuses on commonalities between the plans themselves and the 
requirements as set forth by DCMA and DoD. It studies areas of highest risk in 
performance, schedule, and cost for the suppliers overall and at the service set and One 
Book chapter levels of review. Further, it researches common risk handlings tools 
selected to deal with the various risks identified at the key process/system level of 
planning. 

DCMDW manages more that 125,000 contracts totaling over $500 billion. The 
district consists of 15 field offices on-site at contractor facilities, 13 geographic offices 
handling multiple suppliers for specified areas within the region, and a headquarters 
office in Carson, CA. (DCMDW web site, 2001) As of April 30, 2001 DCMDW’s 
RAMP database population includes 117 strategic assessments and 857 critical plans 
from 5,375 total assessments for the entire region. The strategic and critical assessments 
come from only 27 strategic and 718 critical suppliers respectively. (Shields, 2001) 
Many suppliers have more than one plan due to multiple contracts. 

The sample analyzed here includes 42 plans from eight different geographic and 
in-plant offices in DCMDW. The plans represent 30 different contractor organizations 
and a cross section of facility-wide and contract(s) specific risk management plans. The 
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plans are all from critical and strategic suppliers. Appendix A provides a listing of the 42 
sampled RAMP risk management plans and their associated offices, locations, and 
suppliers. 

B. OVERALL RISK RATINGS & SERVICE SET SUMMARY 

Appendix B presents an overview of the Overall, Service Set, and One Book 

Chapter risk ratings—^high (H), moderate (M or Mod), low (L)—^in performance (P), 

schedule (S), and cost (C) for each of the 42 RAMP plans. Plans without final ratings are 

indicated as in process (IP). 

Table 4.1 provides an overview of the overall risk ratings for performance, 
schedule, and cost for the 42 sampled plans from critical and strategic suppliers. One 
plan listed overall risk ratings as “in process” and is consequently not included in the 
tabulation. 


OVERALL RISK RATINGS 


Mod 

Low 

Performance 

7 

17 

17 

Schedule 

6 

17 

18 

Cost 

5 

8 

28 


Table 4.1. Overall Risk Ratings. 
(Source: Developed by Researcher.) 


Overall risk ratings are system generated, but can be changed at the discretion of 

the rater; however, the method employed on each individual plan is not readily 

discemable by the reader. Often the risk ratings appear to be average assessments based 

upon ratings achieved at the Service Set level, which are in turn driven by One Book 

Chapter risk ratings. However, there are instances where a high risk rating overrides 

what would otherwise be a lower rating due to the significance or severity of a specific 

risk at the One Book Chapter level and its relative importance to the contract, facility, or 
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program overall. The amount of rationale or detail for the assigned risk rating provided 
at this level is often limited and simplistic. Some merely provide scope descriptions even 
when Overall risk ratings are high. Others can be quite thorough and provide sound and 
meaningful summaries for the supporting information that follows, even when no 
significant risk is present. The length of the plans varies as well and there is no clear 
pattern as to this cause. 

1. Performance 

While there was no absolute majority for risk ratings, a significant and equal 
proportion of the plans rated performance both as a moderate and low risk area—40.5%. 
Performance can be viewed as the riskiest area overall, with more high risk ratings than 
schedule and cost, although not significantly so, 16.7% v. 14.3% and 11.9% respectively. 

2. Schedule 

Schedule closely resembles performance risk ratings: 40.5% moderate risk and 
42.9% low. As is often seen through the study, performance and schedule more often 
mir ror each Other due to their close relationship and ultimate control by the contractor. 
Poor or faulty performance will usually result in schedule delays due to additional time 
requirements arising from rework or malfeasance. In the reverse, missed milestones 
(whatever the cause) reflect poorly on contractor performance and can often drive the risk 
rating from this vantage. 

3. Cost 

Cost risk was clearly the area of least risk for the plans overall: 66.7% rated cost 
as a low risk area with 11.9% and 19.1% respectively rating cost as high or moderate. 
Cost can remain isolated from performance and schedule difficulties through Government 
risk mitigation via selection of contract type and payment terms. The Govermnent, being 
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the buyer, has more direct ownership over this area or process than performance or 
schedule and more ability to dictate the final outcome ... at least from a risk management 
perspective. The Government doesn’t perform the service or manufacture the product, 
but it does pay the bills. 

The following sub-chapters delineate risk ratings for each of the One Book 
Chapters under their cognizant service sets. Contract/contractor program/facility specific 
high risk areas are addressed in detail and chosen key processes/systems and their 
associated risk handling tools are discussed. 

C. MAJOR PROGRAM RISK RATING 

The Major Program service set employed in the RAMP database corresponds to 

Chapter 2 of the One Book, Major Program Services. Two of the six subchapters. Earned 
Value Management and Acquisition Logistics, are available for assigning risk ratings in 
RAMP. Eleven (11) of the 42 sampled RAMP plans rated risk areas for one or more of 
the One Book Chapters under this service set. The following risk management plans— 
numbered as per Appendix A—^assigned risk ratings in this area: 6, 10, 23, 24, 30, 33, 
34,38,39,41, and 42. 

The individual ratings for performance, schedule, and cost are automatically 
generated for each RAMP plan based on the input data for all the associated One Book 
Chapter risk ratings for each of these areas. Table 4.2 provides an overview of the Major 
Program service set risk ratings for performance, schedule, and cost of the 42 sampled 
plans fi-om critical and strategic suppliers. Eleven (11) of the sampled plans addressed 
risk management under the Major Program risk area. Thirty-one (31) plans rated this risk 
as not applicable and are not depicted in the table. 
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MAJOR PROGRAM RISK 

msh 

Mod 

Low 

Performance 

0 

3 

8 

Schedule 

1 

5 

5 

Cost 

2 

1 

8 


Table 4.2. Major Program Service Set Risk Ratings. 
(Source: Developed by Researcher.) 


Major Program is the least applied service set among the sampled plans. Only 
26.2% of the plans rank risk in this area and most of the risk was rated low: of the plans 
rating Major Program risk, 72.7% rated performance and cost risk as low, while 45.5% 
rated schedule risk as moderate or low. There were no high risk ratings for performance. 

Of the two assigned One Book Chapters for risk management. Earned Value 
Management was applied twice as often as Acquisition Logistics Support: 23.8% v. 
11.9% due to statement of work (SOW) requirements and Memorandum of Agreement 
(MOAs) between the buyer (program ofiSce) and the local DCMA office. 

Only two plans (#34 and #42) rated high risk at the Major Program service set 
level and both were driven by high risk ratings imder Earned Value Management (EVM). 
The sole high risk rating for Acquisition Logistics Support (#41) was mitigated at the 
Major Program level by a low EVM value in the same area. When applied, EVM seemed 
to take a more prominent role in the risk assignment for the service set as a whole. 

1. Earned Value Management 

The supplier uses an Earned Value Management System (EVMS) to provide 

management information on technical performance, schedule, and cost. They must 

ensure compliance with industry guidelines and contract requirements. As part of its risk 

management efforts, DCMA must provide EVMS system surveillance and program 

analysis to its customers. Table 4.3 provides an overview of the key processes/systems 
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chosen for risk management efforts xmder the Earned Value Management One Book 
Chapter 2.2. 


EARNED VALUE 

MANAGEMENT 

RISK MANAGEMENT 
PLANNO.S 

KEY PROCESSES/SYSTEMS 6 10 23 24 30 33 34 38 41 42 






r 

r 


r 

r 

n 

Accounting 





IB 

■ 


IB 

IB 

IB 

Analysis 








IB 

IB 

IB 

Baselining Changes 


X 






r 

IB 

IB 

Budgeting 








IB 

IB 


Change Incorporation 





IB 

IB 


IB 



Cost Performance Report 




B 

B 

r 





Cost Variance 






IB 

B 




Cost/Schedule Variance 






B 

B 

IB 



Estimate at Completion (EAC) 


Q 

■ 




B 




Earned Value (EV) 










X 

Forecasting 









X 


Indirect Management 





B 






Material Management 





B 






Management Analysis 

■1 

□i 

B 

B 

Bl 



X 



Management Reserve 


■1 

Bl 

Bl 

Bl 

X 





Organizing 




Bl 

Bl 


Bl 

Bl 

Bl 


Schedule Variance 





Bl 

Bl 


Bl 

Bl 


Scheduling 

□1 




X 

Bl 



X 


Subcontract Management 


■1 

Bl 


X 






Training 








Bl 

□1 


Undistributed Budgeting 





Bl 

□1 



Bl 


Use of EV Data 









X 


Work/Budget Authorization 

□1 



■1 

Bl 







Table 4.3. Key Processes/Systems for Earned Value Management. 

(Source: Developed by Researcher.) 

Ten (10) of 42 RAMP plans (or 23.8% of the plans sampled) addressed risk for 
EVMS. The plans focused on 23 different key processes/systems and used ten (10) 
different combinations of processes and systems within EVMS to assess risk for the 
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contractor, facility, or contract in question. As no two plans are alike in the specific 
processes or systems they survey, it is easy to conclude that risk management for EVMS 
is very specific to the contract in question. The most prevalent system chosen for review 
was “Management Analysis”, which was chosen 50% of the time RAMP plans addressed 
EVMS. 


Three plans rated EVMS risk as high in one or more areas. The following details 
the high risk areas specific to the plans indicated and their associated risk handling tools 
chosen to mitigate the risk: 

• #23: The Army Tactical Missile System (ATACMS) Block II TRIP 
(low rate initial production) contract rated under this plan for Lockheed 
Martin Missiles and Fire Control is substantially behind schedule and has 
caused the EVMS schedule area to be rated as high risk. Of the two key 
systems analyzed for EVMS risk, “Subcontract Management” was rated as 
the high schedule risk. Just as contract clauses flow down to 
subcontractors, so does risk management. The major subcontractor has a 
substantial negative schedule variance causing the high schedule risk 
r atin g and additionally driving a moderate risk rating in the cost area due 
to the potential future impact on cost. “Data Analysis” is the selected risk 
hanHlin g tool for “Subcontract Management”: Cost/Schedule Status 
Report (C/SSR) data from bodi the prime and subcontractor is reviewed 
and analyzed monthly to mitigate risk. No further risk handling detail was 
provided. 

• #34: Honeywell’s cumulative cost variance for ten (10) out of (34) 
WBS Item Accoimts is greater than 10% with a wide range from +128% 
to -16%. Program costs are considered likely to increase due to poor 
control over cost variance and threaten to drain the program budget and 
lead to the elimination of required qualification tests. For these reasons, 
cost is rated as a high risk area for EVMS and “Cost/Schedule Variance” 
is the key system reviewed for risk management. Program funding 
depletion also directly affects schedule and performance and drives their 
moderate risk ratings in the EVMS area. “Data Analysis” is the chosen 
risk handling tool for “Cost/Schedule Variance”. Specifically, a 
remaining qualification test will be evaluated weekly until completion. 
Past program test deficiency causes will be reviewed to determine possible 
preventative measures for corrective action. 

• #42: The Raytheon Tucson Evolved Sea Sparrow Missile (ESSM) plan 
rates schedule and cost as high risk areas for Earned Value Management 
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leading to the same risk rating at the Major Program service set level for 
this plan as well. “Earned Value” for two contracts is the key process 
identified for risk management review. One contract rates schedule and 
cost risk high due to a nine-month negative schedule variance (not 
meeting delivery requirements) and a four month negative cost variance 
($50M+ Over Target Baseline). The second contract rates schedule risk 
high due to major slips in key milestones or critical path and high cost risk 
due to unobtainable planned cost targets and regularly unforeseen cost 
events ($50M Over Target Baseline). “Data Analysis” is the chosen risk 
handling tool by and requires bi-weekly reviews of Raytheon Tucson’s 
Cost Performance Report (CPR) along with the Government’s Tec hni cal 
Representative weekly report. 


For each of the plans rating high risk for EVMS, different key processes/systems 
were chosen for risk management focus. The three high risk plans used either a sole 
parameter (#34 and #42) or only two areas to manage risk (#23) while the two plans with 
the greatest number of chosen key processes/systems (#30 and #41) ranked risk low in all 
three areas. “Data Analysis” was the common tool used to mitigate high risk in all 
instances, but different data sources were identified for each of the three high risk plans 
to mitigate risk and seemed appropriate given the differences in the contractor and 
contractual arrangements specific to each plan. The lack of detailed rationale for risk 
handling under “Data Analysis” for “Subcontract Management” (#23) is understandable 
given the indirect relationship of the Government to the subcontractor. The high risk 
areas in #34 and #42 drive a high risk rating at the Major Program service set level. 

2. Acquisition Logistics Support 

DCMA’s policy is to assess the contractor’s ability to meet technical 

performance, schedule, and cost goals for logistics support by reviewing progress on their 

logistics activities and the supplier’s plans, procedures, and reports representative of the 

their logistics management systems/processes. DCMA will identify problem areas and 

recommend Continuous Improvement Opportunities (CIOs) or issue Corrective Action 
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Requests (CARs) to affect process improvements to reduce total ownership cost (life 
cycle cost). Table 4.4 provides an overview of the key processes/systems chosen for risk 
management efforts under the Acquisition Logistics One Book Chapter 2.3. 


ACQUISITION LOGISTICS SUPPORT 

RISK MANAGEMENT 
PLANNO.S 

6 23 30 39 41 

KEY PROCESSES/SYSTEMS 


■ 

■ 

■ 







Cost As An Independent Variable (CAIV) 

■ 

■ 

□ 







Computer Resources Support 



■ 




X 



Depot Level Maintenance Requirements 



□ 




B 



Facilities 



_ 



B 

B 



Logistics Management Plan 



■ 


X 

B 

B 



Log;istics Demonstration 



B 







Maintainability Demonstration 



B 







Maintenance Planning 



B 




X 



Manpower & Personnel 



B 




B 



Packaging & Handling 



fl 




B 



Supply Support 

■ 


B 

fl 



B 



Support Equipment 

■ 


fl 

B 



B 



Supportability Planning 

■ 


B 

B 



B 



Technical Data 

■ 


B 

B 



B 



Training and Support 

u 

u 

X 

U 

— 

__ 


_ 

— 


Table 4.4. Key Processes/Systems for Acquisition Logistics Support. 

(Source: Developed by Researcher.) 

Five of 42 RAMP plans (11.9%) addressed risk for logistics support. The five 
plans used 15 different key processes/systems in four different combinations to assess 
this risk. Given the low identification rate, logistics support does not appear to be 
recognized as a particularly risky area and once identified there were few similarities in 
the systems or processes identified for risk management efforts. The use of the 
“Logistics Management Plan” and “Technical Data” were the two most commonly 
identified key processes/systems for risk management of contractor logistics support. 


used in three of five instances. 
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Only one RAMP plan of the 42 sampled plans rated high risk in this area: 

• #41: Raytheon Tucson Systems plan rated schedule risk high and 

performance and cost risk as moderate. The high risk rating is supported 
by the “Supply Support” and “Support Equipment” key processes/system 
which indicate contractor schedule slippages are due to lack of master 
scheduling. This, in addition to adverse performance trends, drive 
anticipated delays in meeting performance, schedule, and cost objectives. 
100% monthly and quarterly “Data Analysis” is the chosen risk handling 
tool and includes a review of the following data sources: schedule 
analysis, delivery trend analysis program review, root cause data, and cost 
performance data. 


This one high risk plan uses ten different key processes/systems to manage 
contractor risk but most of these areas are not yet actually rated and remain “in process”. 
The chosen key processes are consistent with the Acquisition Logistics Support chapter 
and the risk assignments are adequately supported by rationale and clearly linked with 
each other. The risk handling tool, “Data Analysis’ is consistent with the trend in the 
Major Program risk area and is appropriately detailed in the RAMP plan as to the 
specifics of the data review. The high risk here is mitigated at the Major Program level 
by a lower risk under EVM. 

D. PRODUCT SUPPORT RISK RATING 

The Product Support service set employed in the RAMP database corresponds to 

Chapter 4 of the One Book, Product Performance Services - Right Item. Eight of the ten 

subchapters are available for assigning risk ratings in RAMP: Systems Planning, 

Research, Development and Engineering (SPRD&E), Test and Evaluation Management, 

Configuration Management, Parts Management Program, Software Contract 

Admimstration Services, Supplier Quality Assurance, and Packaging Management 

Program. Two of these subchapters are further broken down: SPRD&E - Design 

Engineering and SPRD&E — Systems Engineering; Supplier Quality Assurance — Quality 

50 



System and Supplier Quality Assurance - Product Quality. Thirty-six (36) of the 42 
sampled RAMP plans rated risk areas for one or more of the One Book Chapters under 
this service set. The following risk management plans—^numbered as per Appendix A— 
assigned risk ratings: 1 - 9, 11 - 26,29 - 31,33 - 39,41, and 42. 

The individual ratings for performance, schedule, and cost are automatically 
generated for each RAMP plan based on the input data for all the associated One Book 
Chapter risk ratings for each of these areas. Table 4.5 provides an overview of the 
service set risk ratings in performance, schedule, and cost of the 42 sampled plans from 
critical and strategic suppliers. Thirty-five (35) of the sampled plans addressed risk 
management under the Product Support risk area. Two plans are “in process” of 
assigning risk ratings and five plans rated this risk as not applicable; these seven plans are 
not depicted in the table. 


PRODUCT SUPPORT RISK 

High 

Mod 

Low 

Performance 

3 

14 

18 

Schedule 

5 

9 

21 

Cost 

2 

11 

22 


Table 4.5. Overview of the Service Set Risk Ratings for Product Support. 

(Source: Developed by Researcher.) 

Product Support is the most applied service set among the sampled plans. A 
strong absolute majority of 83.3% of the plans rank risk in this area. It is the largest area 
with the largest scope from the standpoint of using seven different One Book Chapters 
and nine different risk management areas (two of the chapters being split into two areas). 
Despite the size and potential for risk, given the subject area of the service set, risk 
remained low: 62.9% and 60% of the plans respectively ranked cost and schedule risk as 
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low, while still a clear majority of 51.4% ranked performance risk low and 40% ranked 
performance risk as moderate. 

Of the nine One Book Chapter applications. Supplier Quality Assurance - Product 
Quality was used two to five times as often as any other One Book Chapter level area. It 
was the most commonly used ranking area of any in the RAMP program: 73.8% of the 
RAMP plans addressed risk in this area. Even when only one or two areas are ranked 
under Product Support, Supplier Quality Assurance - Product Quality remains the key 
chosen factor. Given the area’s broad scope and clear application in the post-award 
contract phase of acquisition, this is not surprising. Due to its fiequency of use both 
when few and many One Book Chapters are selected for risk management, it is the key 
driving factor in the overall risk ratings at the Product Support service set level; although 
when include with others, it’s ratings do not seem to out weigh the other applications. 

Only three plans (#17, #20, and #37) rated high risk at the Product Support 
service set level and all three were strongly driven by high risk rankings for Product 
Quality. 

1. SPRD&E — Design Engineering 

SPRD&E surveillance is a risk assessment of the suppliers to conduct systems 
planning, research, development and engineering including engineering systems, 
processes, policies, procedures, practices, activities, and products. The DCMA focus 
here is on design engineering to ensure compliance with contract requirements as 
affecting technical performance, schedule, and life cycle cost. Table 4.6 provides an 
overview of the key processes/systems chosen for risk management efforts xmder the 
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System Planning, Research, Development and Engineering (SPRD&E) One Book 
Chapter 4.1. 


SPRD&E - DESIGN ENGINEERING 
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Table 4.6. Key Processes/Systems for SPRD&E-Design Engineering. 

(Source: Developed by Researcher.) 

Seven of 42 RAMP plans surveyed (16.7%) contractor design engineering efforts 
as part of their product support efforts. The seven plans used nine different key 
processes/systems in seven different combinations to assess this risk. While there was 
clearly a lot of variation in the key processes/systems used by the plans. Engineering 
Management was clearly the most prevalent process identified for risk management, used 
71.4% of the time. 

There were two instances of high risk ratings for design engineering efforts. The 
following details the specifics for the applicable plans and discusses their chosen risk 
handling methods: 

• #30: Aerojet was assigned a high risk rating for design engineering in 

aU three areas of performance, schedule, and cost. “Design 
Analysis/Synthesis” was the chosen key process/system for Government 

53 




surveillance. Improper design requirements could lead to incorrect design 
solutions and/or environmentally hazardous conditions impacting cost, 
schedule, and performance. Although the probability of occurrence was 
only rated moderate, the consequence of occurrence was rated high in that 
failure could likely result in mission failure. The chosen risk handling 
methods were “Surveillance” and “Data Analysis”. Specifically, 
requirements analysis, functional analysis/allocation, and synthesis 
processes were monitored; various activities and metrics were surveyed; 
and policies and procedures were reviewed. The surveillance revealed no 
systemic problems and overall contractor performance was considered 
good, but the current risk ratings and handling methods will remain in 
place due to the high risk of consequence should failure occur. 

• #42: Raytheon Tucson ESSM program rated schedule and cost as high 

risk areas under design engineering. One contract for an Engineering 
Manufacturing Development (EMD) program identified “Engineering 
Management” and “Engineering Planning” as key processes/systems to 
use for risk management. Failure to properly control either area has the 
potential to impact cost and schedule and future transitioning into 
production. A major slip in key milestones and critical path has led to a 
schedule extension. Cost is rated high due to unobtainable cost targets and 
regularly unforeseen cost events; the contract is $50M+ over contract 
value. A second contract for Low Rate Initial Production (LRIP) program 
identified “Producibility” and “Software” as key processes/system to use 
for risk management. A nine month negative schedule variance and 
difficulty in meeting delivery requirements drives a high risk rating for 
schedule. Cost is rated high to the contractor’s failure to contain costs; the 
contract is $50M+ Over Target Baseline (0TB) and has a four month 
negative cost variance. Ratings for “Software” are in process. “Data 
Analysis” was the chosen risk handling tools for the three rated processes: 
The contractors Cost Performance Report (CPR), Cost Schedule/Status 
Report (CSSR), and the Government’s Technical Representative weekly 
report will be reviewed monthly. 

For both of the plans rating high risk for Design Engineering, completely different 
key processes/systems were chosen for risk management focus. Plan #30 focused on 
only one key parameter—^“Design Analysis”, while plan #42 used a mixture of four 
different plans to manage risk. These differences are consistent with the contractual 
efforts and well explained and documented in the rationale. Plan #30 is focused on the 
potential impact of the risk vice its low probability of occurrence (the contractor has 
demonstrated good performance); plan #42 rated risk under two different contracts in 
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different acquisition phases: the engineering based processes were chosen for risk 
management during EMD and production based processes were used during LRIP (the 
contractor is already experiencing some difficulty in fulfilling contractual requirements 
and requires a different focus). “Data Analysis” is once again chosen as a risk mitigation 
tool for both plans, specific to the data for the processes chosen. “Surveillance” is 
additionally used in plan #30 and is used to assess the metrics and processes for systemic 
difficulties. Finding none and given the contractor’s performance, it appears to be a 
worthy task to eliminate probability concerns and focus instead on mitigation of impact 
of failure. The high risk for both plans in the Design Engineering area is not driving high 
risk ratings for the Product Support service set level; each plan uses five other various 
One Book Chapter areas for risk management under Product Support and successfully 
mi tigates the service set level risk rating. 

2. SPRD&E - Systems Engineering 

SPRD«&;E surveillance is a risk assessment of the suppliers to conduct systems 
planning, research, development and engineering including engineering systems, 
processes, policies, procedures, practices, activities, and products. The DCMA focus 
here is on engineering management systems to ensure compliance with contract 
requirements as affecting technical performance, schedule, and life cycle cost. Table 4.7 
provides an overview of the key processes/systems chosen for risk management efforts 
under the Systems Planning, Research, Development and Engineering (SPRD&E) One 
Book Chapter 4.1. 
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SPRD&E - SYSTEMS 

ENGINEERING 

RISK MANAGEMENT 
PLAN NO.S 

KEY PROCESSES/SYSTEMS 22 23 24 26 30 33 35 38 39 41 42 
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Table 4.7. Key Processes/Systems for (SPRD&E)-Systems Engineering. 

(Source: Developed by Researcher.) 

Eleven (11) of 42 RAMP plans (26.2%) surveyed contractor systems engineering 
efforts as part of their product support efforts. The 11 plans used 25 different key 
processes/systems in 11 different combinations to assess this risk. There is clearly a lot 
of variation in the choice of key systems/processes to use for risk management. The most 
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often used were Systems Engineering, Systems Requirements, and Systems Safety, each 
used in 36.4% of the plans. 

There was one instance of high risk rating for systems engineering. The 
following details the specifics of the applicable plan and discusses its chosen risk 
handling method: 

• #42: Raytheon Tucson ESSM program rated schedule and cost as high 

risk areas for systems engineering. “Systems Requirements” was chosen 
as the key process for risk management imder the EMD program contract. 
A major slip in key milestones and critical path has led to a schedule 
extension and the high schedule risk rating. Cost is rated high due to 
imobfednable cost targets and regularly unforeseen cost events; the 
contract is $50M+ over contract vdue. “Data Analysis” is the chosen risk 
handling tool: a bi-weekly review of CPR along with the Government 
Technical Representative weekly report. 

This one high risk plan in this area uses only one key system, “Systems 
Requirements” to manage contractor risk while the other ten less risky plans used an 
average of four key parameters each to mitigate risk in the Systems Engineering area. 
The rationale seems supportive of the assigned rating and consistent with the EMD 
design and integration activities that have experienced technical difficulties. The chosen 
risk handling tool is “Data Analysis” of performance reports applicable to the 
contractor’s requirements. 

3. Test and Evaluation 

The focus here is on the manufacturer’s test engineering/design process and test 
management systems that verify compliance with contract performance requirements. 
DCMA seeks to identify potential test problems and notify customers of suppler test 
decisions. Test data can be an indicator of supplier problems in design, development, 
production, or system deployment. Table 4.8 provides an overview of the key 
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processes/systems chosen for risk management efforts under the Test and Evaluation 
Management One Book Chapter 4.1.1. 


TEST & EVALUATION 

RISK MANAGEMENT 
PLANNO.S 

KEY PROCESSES/SYSTEMS 1 6 22 23 24 26 30 33 34 35 38 39 41 42 
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Table 4.8. Key Processes/Systems for Test and Evaluation. 
(Source: Developed by Researcher.) 


Fourteen (14) of 42 RAMP plans (33.3%) assessed supplier test and evaluation 
performance. Eighteen (18) key processes/systems in 13 different combinations are used 
to assess contractor risk in the 14 plans. There is a great deal of variability in the chosen 
key systems/processes for risk management efforts. However, “Acceptance Tests” were 
used in 35.7% of the plans rating the test and evaluation efforts. No risk areas were rated 
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high. Seven plans ranked risk as low, six ranked risk as medium, and one plan remained 
“in process”. 

4. Configuration and Technical Data Management 

The contractor conducts Configuration and Technical Data Management to 

maintain product design and integrity; control form, fit, and fimction; determine 
engineering and cost tradeoff decisions of technical performance, producibility, 
operability, and supportability; and maintain historic data files. DCMA’s role is to verify 
the contractor’s process has controls for establishing the proper baseline and perform 
necessary reviews and product audits to ensure the contractor’s compliance. Table 4.9 
provides an overview of the key processes/systems chosen for risk management efforts 
xmder the Configuration Management One Book Chapter 4.2. 

Sixteen (16) of the 42 RAMP plans (38.1%) evaluated this area for risk. Thirteen 
(13) key processes/systems in nine different combinations are used to assess contractor 
risk in the 16 plans. While there is a lot of variability between the plans as a whole, 
“Configuration Control” was used in 56.3% of the plans rating configuration and 
technical data management. There were no high risk areas and risk throughout was 
predominantly low: 68.8% ranked performance and schedule risk as low; 75% ranked 
cost risk low. 
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CONFIGURATION & 

TECHNICAL DATA 

MANAGEMENT 

RISK MANAGEMENT 
PLANNO.S 

KEY PROCESSES/SYSTEMS 1 2 3 5 6 22 23 24 26 33 34 35 38 39 41 42 
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Table 4.9. Key Processes/Systems for Configuration and Technical Data 

Management. 

(Source: Developed by Researcher.) 

5. Parts Management Program 

The Parts Management Program is intended to standardize parts to reduce 
inventory and costs for drawings and testing and improve systems commonality, 
interoperability, reliability, standardization, maintainability, and interchangeability. 
DCMA must assess the contractor’s program in this area to account for risk associated 
with noncompliance and possible impact to performance, schedule, and cost. Table 4.10 
provides an overview of the key processes/systems chosen for risk management efforts 
xmder the Parts Management Program One Book Chapter 4.2.1. 

Six of 42 RAMP plans (14.3%) address this risk area. Fourteen (14) key 
processes/systems are used in six different combinations to assess contractor risk in the 
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six plans. “Parts Evaluation/Authorization Process” is the only key process identified as 
a risk managenient area in two separate RAMP plans. There is absolutely no 
commonality here between frequency of chosen processes/systems for risk management 
or overall configuration of the risk management plan. 
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Table 4.10. Key Processes/Systems for Parts Management Program. 

(Source: Developed by Researcher.) 


There was one instance of high risk in Parts Management Program. The 
following details the specifics for the applicable plan and discusses its chosen risk 
handling methods: 

• #22: McDonnell Douglas Helicopter Systems’ Parts Management 

Program was rated as a high performance, schedule, and cost risk. High 
performance risk is due to high consequence of failure, outstanding 
corrective action issues from a previous audit, and past performance 
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instances of similar corrective action difficulties. High schedule risk is 
based on the consequences of process failure and previous contractor 
mitigation factors. High cost risk is present due to consequences despite 
the fact the contractor has adequate processes in place. “Safety of Flight 
Parts” is the key process/system identified for risk management based on 
the safety and mission elements it controls. Monthly “Data Analysis” of 
metrics and 100% inspections of all flight safety part installations are the 
chosen risk handling methods and are conducted dually by DCMA and 
Boeing. 

Parts Management is the least used of the nine areas under Product Support but its 
risk factors for the plans overall do not indicate any nuances different from the other 
areas. The one high risk plan manages contractor risk through the use of one key process, 
“Safety of Flight Parts” which is a proper focal point for mitigating impact of failure, 
which is loss of life in this case. “Data Analysis” of metrics and 100% inspections seems 
appropriate given the nature of the risk and chosen key parameter for risk management. 
There is no significant correlation between risk ratings under this area and those derived 
at the service set level other than contributing factors to the average rating. 

6. Software CAS 

Software Configuration Management Services include software; the supplies, 
processes, procedmes, and activities attributable to software development; software 
documentation; software embedded in test equipment; and non-deliverable software 
products. DCMA assess the contractor’s software development efforts and possible 
performance, schedule, and cost impacts. Table 4.11 provides an overview of the key 
processes/systems chosen for risk management efforts under the Software Contract 
Administration Services (CAS) One Book Chapter 4.3. 

Eleven of 42 RAMP plans (26.2%) provide for contractor surveillance in this 
area. Seventeen (17) key processes/systems are identified and ten combinations used for 
the eleven plans. “Software Configuration Management” is the most frequently used key 
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process/system, applied 81.8% of the time. “Software Quality Assurance” likewise is 
used to a significant degree, 63.6% of the time. There is clearly a lot of commonality and 
congruence between the various plans under Software CAS. 
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Table 4.11. Key Processes/Systems for Software CAS. 

(Source: Developed by Researcher.) 

There is only one incidence of high risk in Software CAS. The following details 
the specifics for the applicable plan and discusses its chosen risk handling methods: 

• #20: Motorola SSG received a high risk rating for schedule in Software 

CAS. This is because Motorola is going to deliver six to eight months 
late. Despite this the cost risk remains low, as this is a fixed price 
contract. “Software Project Planning” is the key process identified for risk 
management; this monitors the contractor’s compliance to the applicable 
software development plan and Statement of Work (SOW) which remains 
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at risk due to inadequately defined interfaces slowing the software 
development and affecting schedule. “CMM Based Insight” (contractor 
monitoring) is the identified risk handling method; this includes a review 
of the contractor’s software development plan (SDP) and the statement of 
work (SOW). 

Most of the risk under Software CAS is low; 70.0% each for performance and 
schedule and 90% for cost. The one plan rating high risk for software (#20) uses only 
one key process for risk management: “Software Project Planning” which is not used 
solely elsewhere, but only in conjunction with other processes/systems to manage 
software risk in totality. It was chosen in this case because it is a Letter of Delegation 
(LOD) task and it appears to be consistent with a subcontracted effort. The high schedule 
risk imder Software CAS for plan #20 is a contributing factor to the high risk rating 
assigned to schedule at the Product Support service set level for plan #20, but as with the 
other plans, it does not seem to be an overriding factor. Monitoring contractor meetings 
(“CMM Based Insight”) as the chosen risk handling tool seems appropriate as well to a 
subcontract effort. 

Current information indicates Program Managers consistently have problems with 
software acquisition in the form of cost overruns, slippage in schedule, and 
nonperformance in terms of meeting specification standards, mission requirements, and 
functionality. (Nissen) In fact, it is often regarded as the highest risk element in weapon 
system development: management is inconsistent or reactive, predictable risks are 
ignored, and quality standards are often traded for schedule, performance, or cost. 
(GSAM 6.4.1.1, 2000) Given this, it is surprising DCMA rates risk in this area so low; 
they may be underestimating the probability or magnitude of the problem should software 
development go awry. 
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7. Supplier Quality Assurance - Quality System 

DCMA performs oversight functions to assess contractor compliance with 
technical, manufacturing, and quality assurance requirements. Due to the breadth of this 
program, multi-functional teams often perform the surveillance to maximize the scope of 
the evaluation and share information within the DCMA Contract Management Office 
(CMO). Quality Assurance activities are - performed whenever inspection and/or 
acceptance at origin is assigned to DCMA unless specifically not required by the 
customer or governed by other policies. For RAMP purposes. Supplier Quality 
Assurance is divided into two areas for risk handling! Quality System and Product 
Quality. 

Quality System audits are performed when directed by the customer, existing data 
is inadequate or unavailable to properly assess the contractor quality assurance system, or 
the contractor’s process has been substantially changed, requiring a new baseline review. 
DCMA measures performance against the DCMA Audit Checklist and International 
Organization for Standardization (ISO) 9000 series quality systems models. The 
contractor is invited to participate in these audits. Table 4.12 provides an overview of the 
key processes/systems chosen for quality system risk management efforts under the 
Supplier Quality Assurance (QA) One Book Chapter 4.4. 

Fourteen (14) of 42 RAMP plans (33.3%) addressed quality systems as a risk 
management area. The plans focused on 26 different key processes/system and used 13 
different combinations of processes and systems within quality systems to assess risk for 
the contractor, facility, or contract in question. The most prevalent systems chosen for 
review were “ISO 9002”, “Design Control”, and “Internal Quality Audits”; each used 
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21.4% of time. As is visibly apparent in the table there is no real commonality or 
congruence between the plans with most key processes/systems only being used once. 
Additionally, one plan, Raytheon Tucson Systems (#41) clearly addresses risk 
management to a degree not even approached in the other 13 plans. 

One RAMP plan rated quality system risk as high. The following details the high 
risk rating in all three areas of performance, schedule, and cost and the associated risk 
handlin g tool chosen to mitigate the risk: 

• #20: DCMA rates performance, schedule, and cost as high risk areas for 

the Motorola SSG plan citing ^ near certainty of complete failure for sub¬ 
system of the F-22 program. Significant instances where there are product 
quality issues for form, fit, and function and resource deficiencies in the 
form of new employees/engineers drive the poor performance rating. The 
schedule for estimated time of delivery has already been extended five 
months beyond purchase order delivery date. The cost is likewise rated 
high, even though this is a fixed-price subcontract due to the high 
probability of unknowns becoming out-of-scope work issues. “Design 
Control” is the key process chosen for risk management and “System 
Evaluation” is the risk handling tool: DCMA QA Representative is to 
attend the bi-weekly meeting with the Quality Assurance Team for 
problem status and schedule impact. 

The one high risk plan uses only one key process/system out of 26 different options used 

throughout the sample for risk management in the Quality System area: “Design 

Control”. All three risk areas of performance, schedule, and cost here drive higher risk 

ratings at the Product Support service set level for plan #20 and high risk for the plan 

overall. This high risk plan is not inconsistent with the other plans applying Quality 

System risk management efforts; 64.3% of the plans use only one key process/system for 

risk mitigation efforts. However, risk in this area generally remains low: 68.8% rate 

schedule and cost risk as low; 56.3% rate performance risk low. The risk tmder plan #20 

however, runs consistent throughout the entire RAMP plan and draws a common thread 

through the other two One Book Chapter areas vmder which it addresses risk 
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management. The chosen risk handling tool is “System Evaluation” and this seems 
consistent with the need for frequent contractor/Govemment interface. 
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Table 4.12. Key Processes/Systems for Supplier QA-Quality System Risk 

Management Efforts. 

(Source: Developed by Researcher.) 
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8. Supplier Quality Assurance — Product Quality 

Each lot of ou^ut jfrom a high risk processes must be sampled using a statistically 
valid sampling plan. CMO personnel have discretion in forming lots for these samples. 
Table 4.13 provides an overview of the key processes/systems chosen for product quality 
risk management efforts under the Supplier Quality Assurance One Book Chapter 4.4. 

Thirty-one (31) of 42 RAMP plans (73.8%) addressed product quality as a risk 
management area. The plans used 140 different key processes/system and 29 different 
combinations of processes and systems within product quality to assess risk for the 
contractor, facility, or contract in question. The most prevalent systems chosen for 
review was “Final Inspection”, used in 23.8% of the plans. While this high percentage 
might seem to indicate DCMA is waiting until the product is finished before making sure 
it is acceptable, the large number of “in process” reviews (i.e. the other 139 key 
processes/systems used to evaluate product quality risk) and the numerous quality 
assurance evaluations of the contractors’ Quality System cited earlier in this chapter 
indicates DCMA’s proactive approach to monitoring quality. 

Five RAMP plans rated product quality risk as high. The following details the 
high risk rating for the identified areas of performance, schedule, and cost and the 
associated risk handling tools chosen to mitigate the risk. 
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Key Processes/Systems for Supplier QA-Product Quality. 
(Source: Developed by Researcher.) 


70 























































#2: Raytheon Electronic Systems was rated with a high performance, 

schedule, and cost risk. Raytheon is currently unable to meet contracted 
delivery schedules due to inadequate manufacturing capability. There was 
no integrated master schedule between three facilities involved in the F/A 
18 program, low yields on some subassembly circuits, a shortage of test 
equipment, and a need for additional employees to increase production 
capacity. Risk mitigation plans were cited as being in place. However, 
the key processes/system identified for risk management do not support 
the most recent rating assignments. “Acceptance Testing”, “Final 
Inspection”, and “Material Review Board” (MRB) all contained the initial 
low risk in all three areas of performance, schedule, and cost and clearly 
had not been updated to support the more current chapter rating. 
“Corrective Action” (as required), “Data Analysis” (collected quarterly) 
and selected “Product Audits” were risk handling methods chosen for the 
“Acceptance Testing” and “Final Inspection” processes. “Product Audits” 
are conducted for “use as is” and “repair” items under the MRB process. 

#14: Westinghouse Electric facility plan was rated as a high 
performance, schedule, and cost risk. Two key processes, 

“Documentation” and “Receiving Inspection” support this risk ratings 
because failures in these processes has resulted in nonconforming material 
delivered to the customer. Additionally, there are numerous contractor 
reorganizations involving up to 50% personnel lay-offs creating serious 
losses in the corporate knowledge base. “Corrective Action” using 
Corrective Action Reports (CARs) issued for contract deviations, “Record 
Review” involving a 100% review of shipment records at final inspection, 
and 100% “Product Audits” of all items presented to the Government for 
acceptance are the selected risk handling tools. 

#17: Stewart and Stevenson were assigned high risk ratings for 
performance, schedule, and cost under Supply Quality Assurance 
Product Quality. “Shipping” is the chosen system for risk management 
review. The contractor has failed to achieve ISO 9000 certification and 
was previously issued a Level III Corrective Action Request for 
deficiencies in their quality management system. On-time delivery was 
75% as result of product quality deficiencies. DCM surveillance and 
audits were suspended imtil the contractor can obtain a repeatable and 
positive Government release quality. “Inspection” is the chosen risk 
hanHIing tool. There were inconsistencies in the rating narrative: “Cost” 
was described as low risk due to fixed-priced contracts, however tiie 
overall, service set, and chapter rating assigned a high rating risk to this 
area. 

#20: Motorola SSG is assigned a high risk rating for performance under 
product quality. It is highly likely there will be a major impact on 
hardware performance due to subcontractor interface specification 
requirements for continued development. “Subcontractor Oversight” is 
the key process identified to manage this risk and no risk handling tools 
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were cited due to the delegated nature of the risk management area. 
Schedule received only moderate risk ratings because the situation is not 
expected to impact current build, but rather those in the future. Cost was 
rated moderate as well, even though this is a fixed-price effort, due to the 
potential for out-of-scope work requirements. No specific risk handling 
tools were annotated, although it was noted that the product quality 
assurance area was constantly monitored by DCMA and any changes 
would be promptly noted and reflected in the risk ratings. 

#22: McDormell Douglas Helicopter Systems effort for the Longbow 
Apache program was assigned a high risk rating in all three areas of 
performance, schedule, and cost for product quality. Joint Aircraft 
Inspection was the first identified key performance parameter and was 
rated as high risk in all three areas: Performance rating indicated that a 
single failure could result in loss of life or total mission failure and 
product technical performance requirements continually fail acceptance 
criteria. Schedule rating was high because failure to repair in a timely 
manner would likely affect the remanufacturing effort. Cost increase was 
considered likely. “Product Audits”: 100% inspection of aircraft when 
ready for inspection, daily was the identified risk handling tool. 
“Maintenance” was the second performance area chosen for risk 
management efforts and was rated as moderate risk in all three areas: 
Performance rating was moderate due to two Corrective Action Reports 
(CARs) being issued in the last year, not all mechanics are fully trained, 
repetitive errors, and aircraft discrepancies noted during customer 
inspection. Schedule risk is moderate because failure to detect 
deficiencies during this process that incorporates delivery preparation of 
aircraft, would impact meeting delivery schedule. High cost risk due to 
failure to detect deficiencies would transfer costs of correction to the 
customer. “Product Audits” of meeting inspection criteria in accordance 
with aircraft maintenance publication is the chosen risk handling tool and 
are conducted on all aircraft: 100% intensity. 

#37: Telechem International Inc. received high risk ratings in all three 
areas of performance, schedule, and cost for the product quality area. Five 
key processes/systems were identified for risk management activity: 1. 
“COA” was rated as high risk in all areas; it is required to be reviewed and 
inspected due to critical application. “Data Analysis” is the chosen risk 
handling tool and will be accomplished with meetings with the contractor 
regarding each contract. 2. “Contract Review” was rated as high 
performance and cost risk and moderate schedule risk. Rationale 
indicated it’s critical application to identify requirements and no further 
detail. The risk handling tool is a “Contract Award Meeting” for each 
contract. 3. “Inspection/Test” shall be performed due to product 
problems. 100% “Product Audits” are the identified risk handling tool 
using Defense Energy Supply Center (DESC) guidance. 4. “Packaging 
and Shipment” received high risk ratings in all three areas due to high 
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failure rate upon receipt at DESC. “Product Audits” as per DESC 
guidelines are the identified risk handling tool. 5. “Purchasing” was 
assigned a high performance risk rating and moderate ratings for schedule 
and cost. These ratings were assigned due to questionable documents that 
were hard to verify. The identified risk handling tool for this area was 
“Process Proofing/Product Audit”. 

The degree of possible variation for key processes/systems is so great for this area 
it is difficult to make any sort of meaningful comparisons. Although, it is clear that the 
plans are not carbon copies of each other and specifically address product quality issues 
for the contractual effort in question and that is entirely appropriate. Product Quality for 
the sampled plans addresses risk management through 140 different key parameters, over 
four times as many as any other areas in the RAMP database. With such a strong 
presence in the risk management data. Product Quality is a key driver in Product Support 
service set level risk ratings and RAMP plan ratings Overall. Performance risk was rated 
moderate 50% of the time, with schedule and cost each ranked as low 53.3% of the time. 
High risk ratings were assigned to these areas only 16.7% and 13.3% of the time, 
respectively. 

Some commonalities can be foimd in the risk handling tools used to mitigate 
Product Quality risk: “Product Audits” and some version of “Test and Inspection” are 
common risk mitigation technique applied to the high risk Product Quality key 
processes/systems, indicating a natural Government propensity to ensure its getting what 
it paid for prior to acceptance. 

9. Packaging Management Program 

DCMA provides packaging assistance and support to its customers to ensure 

adequate packaging performance in accordance with the contractual arrangement and the 

item’s physical characteristics, destination, and use. DCMA support includes 
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surveillance of the contractor’s performance and capability inclu ding availability of 
packaging specification information, adequate handling processes, equipment, and 
packaging costs. The goal is desired protection at the least practical cost to prevent 
deterioration or damage until customer delivery. Table 4.14 provides an overview of the 
key processes/systems chosen for the packaging management program under the 
Packaging Management Program One Book Chapter 4.4.4. 


PACKAGING MANAGEMENT 
PROGRAM 

KEY PROCESSES/SYSTEMS 

RISK MANAGEMENT 
PLANS NO.S 
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Table 4.14. Key Processes/Systems for Packaging Management Program. 
(Source; Developed by Researcher.) 


Seven of 42 RAMP plans (16.7%) addresses the packaging management program 
as a risk management area. The seven plans used five different key processes/systems in 
two different combinations. Clearly, when this area is a chosen area for review, there is 
a great deal of continuity between the plans. This is likely due to DCMA’s policy to 
maintain a Packaging Management Program and provide Packaging Specialists to 
perform fimctions and assist in the packaging process. Such specific guidelines easily 
lend themselves to consistent application throughout DCMA. Three key 
processes/system were used in all plans: “Handling”, “Marking”, and “Packaging”. 
There were no high risk areas identified under the packaging management program. Five 
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plans rank all three risk areas as low, one plan ranks risk as moderate, and one remained 
“in process”. 

E. DELIVERY RISK RATING 

The Delivery service set employed in the RAMP database corresponds to Chapter 
5 of the One Book, Delivery Services - Right Time. Two of the four subchapters are 
available for assigning risk ratings in RAMP: Schedule and Delivery Management and 
Contract Safety Requirements. Thirty-two (32) of the 42 sampled RAMP plans rated risk 
areas for one or more of the One Book Chapters under this service set. The following 
risk management plans—^numbered as per Appendix A—assigned risk ratings: 1,4-9, 
13 - 19,22 - 25, and 29 - 42. 

The individual ratings for performance, schedule, and cost are automatically 
generated for each RAMP plan based on the input data for all the associated One Book 
Chapter risk ratings for each of these areas. Table 4.15 provides an overview of the 
service set risk ratings in performance, schedule, and cost of the 42 sampled plans from 
critical and strategic suppliers. Thirty-two (32) of the sampled plan addressed risk 
management under the Delivery risk area. Ten (10) plans rated this risk as not applicable 
and are not depicted in the table. 


DELIVERY RISK 

High 

Mod 

Low 

Performance 

7 

13 

12 

Schedule 

6 

11 

15 

Cost 

5 

9 

18 


Table 4.15. Overview of the Service Set Risk Ratings for Delivery Risk. 

(Source: Developed by Researcher.) 


Delivery is the second most applied service set among the sampled plans. Thirty- 
two plans or 76.2% of the RAMP plans ranked risk in this area. While the majority of the 
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risk assignments in this area were like elsewhere, low and moderate, this service set had 
the most plans rated as high risk: eight plans or 25% of the plans addressing Delivery 
ranked one or more areas of performance, schedule, and cost as high risk. 

Of the two assigned One Book Chapters for risk management. Schedule and 
Delivery Management was applied four times as often as Contract Safety Requirements: 
69.0% V. 16.7% of the time. 

Eight plans rated high risk at the Delivery service set level and all were driven by 
high risk ratings under Schedule and Delivery Management. The sole high risk rating for 
Contract Safety Requirements was mitigated at the Delivery level by a low Schedule and 
Delivery Management value in the same area. When applied. Schedule and Delivery 
Management seemed to take a more prominent role in the risk assignment for this service 
set as a whole. 

1. Schedule and Delivery Management 

DCMA’s policy is to improve on-time deliveries by reducing delinquency causes 
in the acquisition process, pre-notify customers of potential delays, and respond to 
customer inquiries. These activities assist the customer to meet readiness requirements, 
identify alternative logistic support mechanisms, and select proven performers. Table 
4.16 provides an overview of the key processes/system chosen for schedule and delivery 
risk management under the Schedule and Delivery Management One Book Chapter 5.1. 


76 


SCHEDULES DELIVERY 







MANAGEMENT 









RISK MANAGEMENT PLANS 


IkEY PROCESSES/SYSTEMS 

1 

4 

5 

6 

3 

13 

14 

16 

17 

18 

19 

22 

23 

24 

25 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 





















X 



E 

E 






Contract Review 













X 










m 

m 



















_ 










ID 

ID 

■ 

















X 















_ 



1 ITi'ifil n.f. 

■ 



















X 










Manufacturing Management 













X 















































Material Process Control 










X 



■ 

■ 




■ 

_ 


■ 






















a 

D 




D 

— 


D 














X 














■ 










■ 





D 










X 

■ 

■ 



a 

■ 





■ 

■ 

■ 

■ 

D 


X 


X 



D 

D 

D 

D 

a 

a 

a 



D 

a 


X 

_ 

D 


X 



B 

D 

D 

B 


Production Schedule 





X 














■ 

_ 





























D 

























_ 





_ 

D 










Receive and Inspect 










X 



_ 







X 

_ 











X 











_ 









1 






X 

B 

Services Management Control 
Process 




























1 

1 

Vendor Selection Process 

_ 






_ 

_ 

_ 

X 

■ 

_ 




— 


— 

_ 


_ 


_ 

_ 

_ 


_ 

_1 

_i 


Table 4.16. Key Processes/System for Schedule and Delivery Management. 
(Source: Developed by Researcher.) 


Twenty-nine (29) of 42 RAMP plans (69.1%) addressed schedule and delivery as 
a risk management area. The 29 plans used 19 different key processes/systems in 15 
different combinations to assess risk for the contractor, facility, or contractor in question. 
“Production Planning and Control” was clearly the most commonly used process, used 
62.1% of the time RAMP plans assessed the contractors schedule and delivery system. 

Eight RAMP plans rated schedule and delivery quality risk as high. The 
following details the high risk rating for the identified areas of performance, schedule, 
and cost and the associated risk handling tools chosen to mitigate the risk: 

• #5: Raytheon Electronic Systems FI8 Spare/Support program rated 

Schedule and Delivery Management a high risk in performance and 
schedule and low risk in cost. “Production Planning and Control” was the 
key process/system chosen for risk management efforts. The high risk 
rating for performance is due to the contractor’s full manufacturing 
capacity that is unable to meet delivery schedules required by the contract. 
The contractor initially lacked an Integrated Master Schedule (IMS) 
between three disparate facilities. Now, an IMS is in place and updated 
weekly. Additionally, subcontractors have been added to assist in the 
production effort. Schedule risk is high due to missed delivery milestones 
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because of low yields on components and a shortage of Special Test 
Equipment for the increased delivery schedules. Additionally, employees 
are required to increase production. Cost risk remains low due to Ae firm 
fixed price contracts in place for the program. “Corrective Action” as 
needed and monthly “Data Analysis” and “Product Audits” are the tools 
identified for risk handling measures. 

#13: Graco Industries is given a high risk rating in all three areas of 
performance, schedule, and cost for schedule and delivery management. 
“Production Planning and Control” is the key process/system chosen for 
risk management. The high risk performance rating is due to the 
contractor’s lack of a well managed IMS for in-house Government 
contracts, a high turnover of personnel, and constant lack of capacity. 
Schedule is rated as a high risk because the contractor is on-time less than 
20% of the time. Even though this is a fixed price contract, DCMA 
justifies its high cost risk rating because the supplier is a sole source 
provider and there are “intangible costs” associated with failing to deliver 
on time; the Government lacks other options should they fail to provide 
the items when needed. However, Overall, the contractor is assigned a 
low cost rating, due to the fixed price nature of the contract. Five risk 
handling tools are applied to this problem: 1. “Alerts” - issued each time 
the contractor will miss the final delivery date. 2. “Contract Abstract” - 
complete review each time a new contract or change order. 3. 
“Corrective Action” - issue Corrective Action Request (CARs) as 
necessary. 4. “CPSS Requests” - schedule as needed when received from 
the customer. 5. “Production Person Workload (PPW) Report” - review 
the PPW for past due orders and upcoming orders on a daily basis 

#14: Westinghouse Electric Corporation facility plan rated high risk in 
performance and schedule and moderate cost risk for schedule and 
delivery. “Production Planning and Control” is the area chosen for risk 
management. The high performance rating is due to numerous quality 
problems associated with incomplete data packages and parts not within 
established tolerances. This is likely caused by expedite actions leading to 
circumvention of normal lead times. Schedule risk is rated high: the 
contractor has delivered on-time once in the last two years even after 
receiving many contract modifications for delivery extensions. Two 
CARs have been issued for poor schedule trend performance. Cost risk is 
moderate because all contracts are firm fixed price with no progress 
payment; however, schedule slippages and product reworks lead to 
increased cost risk. “Corrective Action”—issuing CARs and monthly 
“Schedule Reviews”—100% delivery schedules for all contracts. 

#17: Steward & Stevenson’s facility plan was rated high risk in all three 
areas of performance, schedule, and cost for schedule and delivery. 
“Production Planning and Control” is the area chosen for risk management 
because it is the top level system that controls the contractor’s ability to 
satisfy the delivery schedule. Performance risk is rated high because the 
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contractor does not seem to have a well managed IMS for Government 
contracts. Many expedite actions are required and vendor control is 
lacking. The contractor’s less than 50% rate for on-time delivery drives 
the high schedule risk rating. Lack of production planning results in 
numerous delayed shipments. Cost risk is high due to expediting efforts. 
Five risk handling tools are used for this area: 1. “Contract Abstract”- 
complete review each time a new contract or change order. 2. 
“Corrective Action” - issue Corrective Action Request (CARs) as 
necessary. 3. “CPSS Requests” - schedule as needed when received from 
the customer. 4. “Product Audits” - per shipment each time the 
contractor will miss the final delivery date. 5. “Production Person 
Workload (PPW) Report” - review the PPW for past due orders and 
upcoming orders on a daily basis. 

#19: Davies Rail & Mechanical assigned high risk ratings in all three 
areas of performance, schedule, and cost for schedule and delivery 
management. “Production Planning and Control” was the chosen key 
process/system for risk management. The contractor did not have a well 
managed IMS, the on-time delivery rate is less than 50%, and expediting 
efforts negatively impact the cost. Five risk handling tools were chosen to 
mitigate the risk: 1. “Alerts” - each time contractor will miss the final 
delivery date. 2. “Contract Abstract” - complete review each time a new 
contract or change order. 3. “Corrective Action” - issue Corrective 
Action Request (CARs) as necessary. 4. “CPSS Requests” - schedule as 
needed when received from the customer. 5. “Production Person 
Workload (PPW) Report” - review the PPW for past due orders and 
upcoming orders on a daily basis. 

#22: McDonnell Douglas Helicopter Systems’ Longbow Apache 
program received a high risk rating in the area of performance. 
“Forecasting” and “Production Planning and Control” are the identified 
key process/systems for risk management. “Forecasting” performance, 
schedule, and cost were all rated as moderate risks. Performance under 
“Production Planning and Control” was rated high: High turnover of 
subcontractors supplying critical and flight safety parts have caused 
numerous tooling and drawing changes and increasing probability that 
performance, schedule, and cost objectives will not be met. “Data 
Analysis” is the risk handling tool chosen for this area. 

#41: Raytheon Tucson ESSM program received a high risk ratings for 
performance and schedule and a low risk for cost under Schedule and 
Delivery Management. “Production Planning and Control” and “Schedule 
and Delivery Management” are the two key processes/systems chosen for 
risk management. Performance risk for these areas stems from the 
contractor’s rescheduling of major programs 26 times in the last 12 
months. The operational Master Performance Schedule (MPS) schedule 
metric fluctuates between 70-80%; internal goals have never been met. 
The schedule rating is driven by the additional factor of only a 66% on- 
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time delivery rate. Moderate cost rating stems from the contractor’s use of 
the MRP system as a material ordering system and manag ing by 
workarounds or expediting which negatively impacts cost. Monthly “Data 
Analysis” and “MMAS Meetings with Contractor” were the selected risk 
handling tools for “Production Planning and Control”: analyze processes 
with the contractor to identify root causes and request contractor take 
corrective action on this system. Risk handling tools for “Schedule and 
Delivery Management” are monthly “Data Analysis” and “Root Cause 
Analysis”: review the on-time delivery report and outs tanding 

delinquency report to identify root causes. 

#42: Raytheon Tucson Evolved Sea Sparrow Missile (ESSM) prog ram 
rated high cost risk for schedule and delivery with moderate performance 
and schedule risk assignments. Two key processes/systems were chosen 
for risk review: 1. “Product Development” assigned cost a high risk 
rating due to cost overruns on one contract and obsolete material issues on 
another under the program. 2. “Services Management Control Process” 
assigned a high risk rating to cost due to new requirements potentially 
causing a delayed or missed milestone which may result in costly 
rebaselining activities. The chosen risk handling tools for CESR are 100% 
“Contract/Modification Review”, monthly “Data Analysis”, weekly 
“Meetings”, and monthly “Root Cause Analysis”. 


Seven of the eight plans rating high risk for Schedule and Delivery Management 

identified “Production Planning and Control” as a key process to manage risk; it was the 

sole process for six of the plans. Plan #42 is the exception because the difficulties do not 

seem to stem from the manufacturing process itself This is consistent with the risk 

management efforts in this area across the board that often chose this process as a key 

performance parameter for risk management. A common theme running through these 

plans is the contractor’s lack of an integrated master scheduling plan between facilities or 

within the plant to distinguish Government contract efforts (#5, #13, #17, #19, and #41). 

Other plans focus on various subcontractor difficulties (#22), quality difficulties (#14), or 

obsolescence and new requirements issues (#42). In all cases, the high risk ratings in this 

area drive the risk rating at the Delivery service set level. The chosen risk handling tools 

for these areas seemed common within the specific DCMA office responsible for contract 
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administration: The DCM San Antonio office tended to use “Alerts”, “Contract 
Abstracts”, “CARs”, “CPSS Requests”, and “PPW Report” to handle risk. Raytheon 
offices tended more towards “Data Analysis”, “Root Cause Analysis”, and “Meetings”. 

2. Contract Safety Requirements 

When contract requirements dictate specific safety requirements involving 
Ammunition and Explosives (A&E), Flight Ground Operations, Industrial Operations, 
Into-Plane Refueling Operations, or Maritime Operations, DCMA will evaluate 
contractor high risk operations in accordance with the DCMA Contract Safety Program. 
Table 4.17 provides an overview of the key processes/systems chosen for contract safety 
reqizirements under Contract Safety Requirements One Book Chapter 5.3. 


CONTRACT SAFETY 
REQUIREMENTS 

KEY PROCESSES/SYSTEMS 

RISK MANAGEMENT 
PLANS NO.S 

7 9 15 30 3135 38 
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Table 4.17. Key Processes/Systems for Contract Safety Requirements. 
(Source: Developed by Researcher.) 


Seven of 42 RAMP plans (16.7%) addressed contract safety requirements as a 
risk management area. The six plans used six different key processes/systems in two 
configurations to assess risk for the contractor. “Safety Program” was the system used in 
all instances and clearly drives the high degree of consistency in this area. 
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Only one RAMP plan rated safety risk as high. The following details the high 
risk rating and the associated risk handling methodology to mitigate the risk: 

• #30: Aerojet General Corporation rates performance as a high risk area 

tmder contract safety requirements. “Safety Program” is the chosen key 
process/system to manage risk. Performance received a high rating due to 
the inherent risk of explosives handling operations. Any failure to follow 
safety requirements would significantly increase the severity of mishaps. 
Although no specific risk handling tool was listed, risk handl ing detail 
indicated a Contract Safety Specialist would review the contractor’s safety 
program along with the contractor’s safety representative including 
operational sites, production areas, and subcontractor compliance. 


The one plan with a high risk rating used the same key system to manage risk as 

the other plans imder Contract Safety Requirements: “Safety Program”, selected because 

it is the overarching key process under this area. The rating rationale is sound, based on 

the potential impact vice the probability of the risk event. For this reason, it is reasonable 

that a low risk rating for Schedule and Delivery Management mitigates the high risk 

rating here for the service set. While not specifically spelled out, risk handling will be 

accomplished by reviewing the contractor’s safety program. 

F. BUSINESS AND FINANCIAL SYSTEMS RISK RATING 

The Business and Financial Systems service set employed in the RAMP database 

corresponds to Chapter 7 of the One Book, Business & Financial Systems Services. Four 

of the six subchapters are available for assigning risk ratings in RAMP: Contract 

Property Management, Contractor Estimating System Reviews, Material Management 

and Accounting Systems, and Cost Accounting Standards (CAS) Administration. 

Nineteen (19) of the 42 sampled RAMP plans rated risk areas for one or more of the One 

Book Chapters under this service set. The following risk management plans—^numbered 
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as per Appendix A—^assigned risk ratings: 1, 8, 10, 18, 22, 25, 27, 28, 30, 31, 33 - 36, 
and 38 - 42. 

The individual ratings for performance, schedule, and cost are automatically 
generated for each RAMP plan based on the input data for all the associated One Book 
Chapter risk ratings for each of these areas. Table 4.18 provides an overview of the 
service set risk ratings in performance, schedule, and cost of the 42 sampled plans from 
critical and strategic suppliers. Nineteen (19) of the sampled plan addressed risk 
management imder the Business and financial Systems risk area. Twenty-four (24) plans 
rated this risk as not applicable and are not depicted in the table. 

Business and Financial Systems are used for risk management efforts in 45.2% of 
the sampled plans. The majority of the risk was rated low: 63.2% of plans rated 
schedule and cost risk as low, while 47.4% (still the largest proportion) rated 
performance risk as low. High risk ratings were rare: twice for performance and cost 
and one for schedule. This is undoubtedly indicative of the fixed price contract types 
used commonly throughout the plans. 


BUSINESS & 

FINANCIAL SYSTEMS 

RISK 


Mod 

Low 

Performance 

2 

8 

9 

Schedule 

1 

6 

12 

Cost 

2 

5 

12 


Table 4.18. Overview of the Service Set Risk Ratings for Business and Financial 

Systems Risk. 

(Source: Developed by Researcher.) 

Risk ratings were fairly well dispersed among the four One Book Chapters used 

for the Business and Financial Systems service set with over half the plans using this 
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service set assigning risk ratings for Contractor Estimating System Reviews (CESRs) 
and Material Management and Accounting Systems (MMAS) (63.2%), Contract Property 
Management (73.7%), and Contractor Purchasing System Reviews (73.7%). 

Only three plans (#25, #41, and #42) rated high risk at the Business and Financial 
service set level and all three were driven by high risk ratings for Contract Property 
Management which seems to be the most risky area and a key driver in this service set. 
CESRs provided the other sole high risk rating and contributed to high risk service set 
rating (#41). 

1. Contractor Estimating System Reviews 

Contractor Estimating System Reviews (CESRs) review the contractor’s 
processes of collecting and building cost estimates. The Government must ensure this is 
done according to standards with the right information source and the right system to 
produce reliable and consistent cost information representative of actual costs. Table 
4.19 provides an overview of the key processes/systems chosen for CESR risk 
management under the Contractor Estimating System Reviews One Book Chapter 7.3. 

Twelve (12) of 42 RAMP (28.6%) plans address CESRs as a risk management 
area. The 12 plans use nine different key processes/systems in eight different 
combinations to assess risk for the contractor, facility, or contract in question. “Forward 
Pricing” was the most often used process, identified as key area 66.7% of the time, vrith 
“Cost Accounting System (CAS)” and “Proposal Development” following closely, in use 
58.3% of the time. Combined, these clearly lend some continuity to the risk management 
process. 
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CONTRACTOR 

ESTIMATING SYSTEM 

REVIEW 

RISK MANAGEMENT 

PLANS NO.S 
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Table 4.19. Key Processes/Systems for Contractor Estimating Systems Review. 

(Source: Developed by Researcher.) 

Only one RAMP plan rated CESRs as a high risk area: 

• #41: Raytheon Tucson Systems plan was rated as a high risk in the area 

of cost for CESRs with performance and schedule being assigned 
moderate ratings. Three key processes/systems were chosen for risk 
management: “Cost Accounting System”, “Forward Pricing”, and 

“Proposal Development”. All three areas were individually rated as 
moderate and do not support the high cost risk rating at the One Book 
Chapter level. The chapter narrative indicates a corrective action in place 
for inadequate and late subcontract cost/price analysis. 


The lack of supporting ratings at the key process/system level is inconsistent with 
the scheme of risk management in the RAMP system which is designed to build up from 
the lowest levels of key processes/systems through to an Overall rating supported and 
documented by the ratings at the lower echelons. Additionally, the lack of supporting 
information at these levels means there is no direct correlation between the three chosen 
processes/systems (and their associated risk handling tools) and problem at hand. 
However, high risk under Contract Property Management lends additional credibility to 
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the resulting high risk at the Business and Financial Systems service set level. The three 
chosen systems for risk management are the three most commonly used areas for risk 
assessment under CESR. 

2. Material Management and Accounting Systems 

The concept of Material Management and Accoxmting Systems (MMAS) is to 

ensure that material used to manufacture a product is charged or costed, in the right 
amoimt, to the contract for that product and no other. Suppliers may have munerous 
contractors, both Government and commercial and properly assigning material costs to a 
contract can be a complex and confusing enterprise. DCMA must apply risk 
management to ensure confidence in allowable and allocable material costs assignable to 
a contract. Table 4.20 provides an overview of the key processes/systems chosen for 
MMAS risk management under the Material Management and Accounting Systems One 
Book Chapter 7.5. 

Twelve (12) of 42 RAMP plans (28.6%) addressed MMAS as a risk management 
area. The 12 plans used 15 different key processes/systems in 10 different configurations 
to assess risk for the contractor. “Accounting System Reviews” was the system used 
most commonly, 66.7% of the time. No RAMP plans rated risk as high in this area and 
the predominant risk ratings were low for performance, schedule, and cost. 75%, 91.6%, 
and 83.3% of the plans addressing MMAS. 
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MATERIAL MANAGEMENT & 
ACCOUNTING SYSTEM (MMAS) 

RISK MANAGEMENT 
PLANS NO.S 

1 22 28 30 31 33 34 35 36 38 40 41 
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Table 4.20. Key Processes/Systems for Material Management and Accounting System 

(MMAS). 

(Source: Developed by Researcher.) 

3. Contract Property Management 

Contractors must have an adequate system to manage Government property in 
their possession. Their property control system must serve to control, protect, preserve, 
maintain, and establish accoimtability over Government property. DCMA oversight 
includes activities to assess the contractor’s system to determine priority, degree, and 
level of surveillance required; validate a contractor’s self oversight program; perform 
property administration functions; and investigate loss, damage, or destruction (LDD) of 
Government property. Table 4.21 provides an overview of the key processes/systems 
chosen for risk management rrnder the Contract Property Management One Book Chapter 
7.1. 
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Fourteen (14) of 42 RAMP plans (33.3%) addressed contract property 
management as a risk management area. The 14 plans used 16 different key 
processes/systems in five configurations to assess contractor risk. Eight of the plans 
(57.1%) used the same configuration and 12 plans used Property Management as a key 
processes (85.7%), providing a significant amount of contimxity between the plans. Even 
among five different combinations there is significant congruence/overlap between the 
chosen processes/systems. 

Five RAMP plans rated property management as a high risk area. The following 
details the high risk rating for the identified area of performance, schedule, and cost and 
the associated risk handling tools chosen to mitigate the risk: 
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Table 4.21. Key Processes/Systems Chosen for Contract Property Management. 

(Source: Developed by Researcher.) 
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#10: Brown & Root Services Corporation facility is rated as high risk in 
all three areas of performance, schedule, and cost. Fifteen different key 
processes/systems are used to manage risk, seven of these areas are rated 
as high risk: “Acquisition”, “Contractor Property Close-out”, 

“Disposition”, “Property Management”, “Records”, “Reports”, and 
“Subcontractor Control”. Performance risk is high due to the variety and 
geographic dispersion of Government property under the contractor’s 
cognizance: over 95,690 line items scattered throughout the Balkans 
(Bosnia, Croatia, Hungary, Macedonia, and Kosovo) and valued at over 
$293,046,299. Schedule risk is high due to potential impact of systemic 
deficiencies on the contractor’s ability to order materials and issue 
subcontracts to meet the numerous and varied requirements. Cost risk is 
high due to the cost reimbursable contracts and performance in multiple 
locations. System Evaluation is the chosen risk handling tool to mitigate 
risk for this contractor: an annual property control system audit. 

#25: DRS Infixed Technologies LP M1A2 Abrams Upgrade program is 
assigned a high risk rating in the area of performance with low risk ratings 
for schedule and cost. The contractor’s performance criteria are divided 
into three sub-elements: inherent, property control system, and property 
control system changes. Property control system and property control 
system changes are assigned a high rating because the contractor is new 
and acceptable property control procedures have not been submitted and 
potential changes are unknown. There are no known deficiencies now that 
could impact cost or schedule. “Property Management” is the chosen key 
system for risk management. “System Evaluation” using annual sampling 
is the chosen risk handling tool. 

#40: Raytheon Tucson AMRAAM program is assigned a high risk 
rating for performance, schedule, and cost for contractor property 
management. This risk rating is not broken down into the specific 
elements but cites excessive Lost, Damaged, and Destroyed (LDD) 
property on the program as rationale for the “Property Management” key 
process. “System Evaluation” using an annual sample is the chosen risk 
handling tool. 

#41: Raytheon Tucson Systems is assigned high risk ratings for 
performance, schedule, and cost. Fifteen different key processes/systems 
are used to manage risk, six of these areas are rated as high risk: 
“Acquisition”, “Movement”, “Property Management”, “Records”, 
“Subcontractor Control”, and “Utilization”. The contractor’s property 
system is rated as “unsatisfactory” but “approved” and the contractor is 
pursuing an approved corrective action plan (CAP) and joint audits with 
DCMA personnel to improve their “internal” ratings from “RED” to 
“YELLOW”. “Annual Statistical Sampling” is the chosen risk handling 
tool in all instances. 
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• #42: Raytheon Tucson Evolved Sea Sparrow Missile (ESSM) was 

assigned high risk ratings in all three areas of performance, schedule, and 
cost for property management. “Property Management” is the chosen key 
system for risk management: Performance is rated high due to the 
contractor’s unsatisfactory performance during a property control system 
audit. Process integrity is compromised due to improper management and 
control of Government property and may impact schedule. Problems with 
the system cause overall cost increases. An aimual systems analysis using 
100% “Judgement Sampling” is the chosen risk h an dling tool. 

The five plans rating high risk for Contract Property Management all had a 
common key process/system in common: “Property Management”. Two of the plans 
(#10 and #41) used 15 different key processes/systems to assess risk for the contractor; 
three of the plans (#25, #40, and #42) used only one key parameter for this purpose. Four 
of the plans (#10, #40, #41, and #42) rated all three areas of risk, performance, schedule, 
and cost as high risk, while one plan (#25) rated only performance as a high risk, 
schedule and cost risk remain low. However, despite the preponderance of high risk 
ratings in this area, these factors, when encompassed with the ratings for the other three 
One Book Chapter level ratings, contributed to moderate risk ratings and only led to three 
plans with high risk ratings at the service set level. “System Evaluation” was the chosen 
risk management tool in three instances (#10, #25, and #40), with “sampling” tools used 
in the remaining two plans (#41 and #42). However, “System Evaluation” is conducted 
using annual sampling, which blurs the distinction between the two tools. 

4. Contractor Purchasing System Reviews 

Contractor Purchasing System Reviews (CPSRs) involve the Administrative 
Contracting Officer’s (ACO’s) consent for the prime to place subcontracts and approval 
of the contractor’s purchasing system. When the prime contractor awards subcontracts 
non-competitively or the contract allows all subcontract costs to flow up to the 
Government, the Government is placed at risk. Purchasing system approval allows 
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contracting officers to waive subcontract advance notifications and/or consent to 
subcontract actions and provide early CAS uiforiiiation to base source selection decisions 
and negotiation positions for profit/fee. Table 4.22 provides an overview of the key 
processes/system chosen for risk management under Consent to Subcontract/Contractor 
Purchasing Review One Book Chapter 7.4. 

Nine of 42 RAMP plans addressed contractor purchasing systems as a risk 
management. The nine plans used 11 different key processes/systems in eight different 
configurations to assess risk for the contractor. “Best Value” and “Internal Purchasing 
System Audit” were the two most conunonly used systems/processes for risk 
management: 55.6%. No RAMP plans rated high risk in this area. The majority of the 
risk ratings were low: 66.7% for schedule and 55.5% each for performance and cost. 
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Table 4.22. Key Processes/System for Contractor Purchasing System Review. 
(Source: Developed by Researcher.) 
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G. PAYMENT AND FINANCIAL MANAGEMENT RISK RATING 

The Payment and Financial Management service set employed in the RAMP 

database corresponds to Chapter 9 of the One Book, Payment and Financial Management 
Services. Three of the six subchapters are available for assigning risk ratings in RAMP: 
Progress Payments, Performance Based Payments, and Public Vouchers. Twenty-one 
(21) of the 42 sampled RAMP plans rated risk areas for one or more of the One Book 
Chapters under this service set. The following risk management plans—^numbered as per 
Appendix A—^assigned risk ratings: 1,4-6,19,22 - 25,28,30 - 35, and 38 - 42. 

The individual ratings for performance, schedule, and cost are automatically 
generated for each RAMP plan based on the input data for all the associated One Book 
Chapter risk ratings for each of these areas. Table 4.23 provides an overview of the 
service set risk ratings in performance, schedule, and cost of the 42 sampled plans from 
critical and strategic suppliers. Twenty-one (21) of the sampled plans addressed risk 
management imder the Payment and Financial Management risk area. Twenty-one (21) 
plans rated this risk as not applicable and are not depicted in the table. 

Payment and Financial Management was applied by half of the sampled RAMP 
plans. An absolute majority of the risk at the service set level was rated low: 76.2% for 
performance and schedule and 71.4% for cost. Only one plan (#23) ranked high risk at 
the service set level. 
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Table 4.23. Overview of the Service Set Risk Ratings for Payment and Financial 

Management Risk. 

(Source: Developed by Researcher.) 

Of the three assigned One Book Chapters for risk management, Progress 
Payments Based on Costs and Public Voucher were applied most often (61.9% and 
76.2% respectively), while Performance Based Payments was used less than half the time 
risk was rated, 42.9%. 

Only two plans used any high risk ratings and these were in the Performance 
Based Payments area (#23) which ultimately drove high risk ratings at the service set 
level and in the Public Vouchers area (#42) which was mitigated to moderate risk ratings 
at the service set level by low and moderate ratings in the other two One Book Chapters. 

1. Progress Payments Based on Cost 

Progress payments recognize a contractor’s need for working capital due to long 
lead times and work in process costs and thus provide interim financing for contracts 
other than cost-reimbursement arrangements. DCMA’s role regarding the management 
of progress payments is three-fold: ensure that Government funds are protected, that the 
contractor is paid in a timely fashion commensurate with the actual work performed as 
per contractual requirements, and that overpayments are avoided. To do this the 
contractor’s management systems, fin ancial condition, and contract performance must be 
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monitored. Table 4.24 provides an overview of the key processes/systems chosen for risk 
management rmder the Progress Payments One Book Chapter 9.2. 

Thirteen (13) of 42 RAMP plans (31.0%) addressed progress payments as a risk 
management area. The 13 plans used five different processes/systems in seven different 
configurations to assess risk for the contractor. “Management of Company Financial 
Condition” was the most frequently used process, 84.6% of the time. Clearly there is a 
lot of continuity between the various plans, with a majority of the plans using the exact 
same configuration. No one plan used a key process or system not used elsewhere. No 
RAMP plans rated high risk in this area. An absolute majority of risk ratings were low: 
84.6% for performance, 92.3% for schedule, and 76.9% for cost. 
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Table 4.24. Key Processes/Systems for Progress Pa 3 nnents. 

(Source: Developed by Researcher.) 

2. Public Vouchers 

Contractors submit interim and final public vouchers for costs and fees under 
cost-reimbursement, time-and-materials (T&M), and labor-hour (LH) contracts. DCMA 
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contract auditors are authorized representatives of the Administrative Contracting Officer 
(ACO) for receiving vouchers, approving interim vouchers, authorizing contractor direct 
submission to the disbursing office for those suppliers with approved billing systems, and 
forwarding final payment vouchers to the ACO for approval. The auditor may be the 
Defense Contract Audit Agency (DCAA). Table 4.25 provides an overview of the key 
processes/systems chosen for risk management under the Public Vouchers One Book 
Chapter 9.4. 

Seventeen (17) of 42 RAMP plans (40.5%) addressed public vouchers as a risk 
management area. The 17 plans used six different key processes/systems in ten different 
combinations to assess risk for the contractor. “Management of Voucher Preparation and 
Submittal” was the most commonly chosen area for risk management, used 82.4% of the 
time. One again there is visibly a great deal of continuity between the plans and their 
chosen methodologies. A good deal of overlap exists even when there is variation. 
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Table 4.25. Key Processes/Systems for Public Vouchers. 
(Source: Developed by Researcher.) 
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Only one RAMP plan rated risk as high in this area: 

• #42: Raytheon Tucson ESSM program was rated as high risk in the 

areas of schedule and cost, performance risk was rated moderate. Four 
key processes/systems are chosen to manage risk, as indicated above. 
Only the “Management of Costs” process received high risk ratings: This 
area was chosen because only allocable, reasonable costs are allowable. 
Rating rationale indicated that the contractor had notified the Government 
that additional fimds are needed; past performance indicates previous cost 
control problems; and Limitation of Cost/Limitation of Funds notifications 
were not being provided to the Government until requested and then, late. 
Performance progress consistently lags fimding. The chosen risk handling 
tool for this area is 100% monthly “Audit Voucher for Fee”. 

The key processes/systems chosen for risk management were consistent and used 
extensively by other plans addressing risk management under the Payment and Financial 
Management service set. But only one of the three rated risk as high: “Management of 
Costs”. The rationale for this clearly justified the risk rating and its importance. The 
high risk rating here however, did not produce a high risk rating at the service set level 
due to the mitigation effect of other One Book Chapter risk ratings for the plan. The 
chosen risk handling plan, “Audit Voucher for Fee” seems consistent with the need to 
verify costs. 


3. Performance Based Payments 

Performance based payments provide contractor financing vice payment for 
accepted items. It is applicable when objective and quantifiable performance 
measurements exist or when completion of definable events is appropriate. DCMA is 
responsible for administrating payments imder this program which is preferred due to the 
clearly, definable links between performance and dollars and allowance for the 
establishment of clear goals. Performance Based Payments are used for fixed price type 
contracts when no other financing is provided for. Table 4.26 provides an overview of 
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the key processes/systems chosen for risk management under the Performance Based 
Payments One Book Chapter 9.3. 

Nine of 42 RAMP plans (21.4%) addressed Performance Based Payments as a 
risk management area. The nine plans used seven different key processes/systems in 
seven different configurations to assess risk for the contractor. “Management of 
Performance Based Requirements, Preparation, and Submittal” was by far the most 
commonly used process, occumng 77.8% of the time. RAMP plan #34 was in process 
of rating risk for this area and provided no key process/system information. 


PERFORMANCE BASED PAYMENTS 

RISK MANAGEMENT 
PLANS NO.S 
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X 

X 

X 


X 

X 


X 


Table 4.26. Key Processes/Systems for Performance Based Payments. 

(Source: Developed by Researcher.) 

Only one RAMP plan rated risk as high for performance based payments: 

• #23: Lockheed Martin Missiles and Fire Control was rated as high risk 

in all three areas of performance, schedule, and cost. “Management of 
Performance Based Payment Requirements, Preparation, and Submittal” is 
the chosen risk management area. Physical verification reviews indicated 
delinquent subcontractors. Production lines are co-mingled with other 
contract efforts. Contractor is financially stable, however a potential 
merger may affect this. A moderate number of changes is anticipated to 
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require renegotiation of events and anticipated major changes may expose 
the contract to greater risk than previously known. The chosen risk 
handling tools for this process are 100% monthly “Data Analysis”, 100% 
annual “DCAA Audit” of incurred costs to verify performance payments 
are not advance payments, initially “Established Surveillance Plan”, 100% 
“Product Audits” as submitted for approval, and monthly 100% “Review 
of Paid Vouchers”. 

The high risk plan uses only one key parameter to manage risk for the contractor 
under Payment and Financial Management: “Management of Performance Based 
Requirements, Preparation, & Submittal.” This system is commonly used in the other 
plans under this service set, but usually in conjunction with other key parameters. The 
high risk ratings here drive the high risk ratings at the service set level. The rationale 
noted several problem areas to defend the high risk rating. Five different risk handling 
tools are listed and would seem to encompass the problem through their combined 
umbrella effect regarding nearly every aspect of this area. 

H. VARIABILITY IN THE SAMPLED PLANS 

The following sections address a few summary observations regarding variability 

in the RAMP plans as a whole: 

1. Missing Risk Ratings 

The risk rating process for the RAMP plans start with risk planning. P lanning 
involves the identification of key processes/systems that can have a significant adverse 
affect on performance, schedule, or cost if not properly controlled. The “significant 
adverse affect” is defined based on probability of occurrence and impact. Those 
processes or systems meeting tins requirement as determined by DCMA personnel are 
rated in the RAMP plan. Contract requirements, memorandums of agreement (MOAs) 
with the customer, or other delegations such as subcontract work provide additional 
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reasons to rate specific processes or systems not otherwise identified as a ratable item by 
DCMA. 

The RAMP plans typically indicate why a process or system is chosen to be 
worthy of inclusion but usually make no statement regarding the exclusion of a particular 
Service Set or One Book Chapter or process/system. One Book Chapter ratings are built 
fi-om the rated key processes/systems. So, if no key processes/systems are identified 
under a particular One Book Chapter, then that chapter is not included in the RAMP plan. 
The same is true for Service Set risk assignments, whose ratings follow those given to 
appropriate One Book Chapters. If no chapters under a given Service Set are rated, then 
that Service Set is rated as “NA” or “not applicable”. 

So, while every RAMP plan contains an Overall rating for the contractor, 
contract, program, or facility in question, the plans do not contain ratings for every 
identified Service Set and One Book Chapter formatted into the RAMP database. Absent 
the knowledge of the decisions made during the risk planning phase, consumers of 
RAMP data may not have a full understanding of why individual areas were not 
evaluated. 

2. Key Process/System Choice 

Functional specialists or process owners identify key processes/systems from 
which the risk ratings will be assigned. There is no one reference or master laundry list 
from which these processes may be selected. Specialists refer to a number of gmdes, 
publications, directives, and other information sources specific to their One Book Chapter 
area to determine possible process or system areas conducive to risk management 
surveillance. 
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A good starting place identifying key processes/systems is the DLAD 5000.4 One 
Book. Each One Book Chapter provides some direction regarding the risk planning 
process, but details and data references vary greatly from listing recommended (and 
sometimes required) processes or systems for surveillance to referencing FAR/DFAR 
requirements or DoD Directives. Often data links are provided to specific guides and 
publications e.g. EVMS Guidebook, DSMC Acquisition Logistics Guide, DSMC Test 
and Evaluation Guide, Software Engineering Institute’s Software Capability Maturity 
Model, and the Contractor Purchasing System Review (CPSR) Guidebook. The Supplier 
Risk Management One Book Chapter 3.0 is often referred to as well. 

All One Book Chapters refer the specialist to the applicable contract, appropriate 
modifications, MOAs, and Letters of Delegation/Instruction (LODs/LOIs) for their 
specific RAMP plans. Given the contract/contractor specific requirements as well as the 
previously mentioned data sources for key process/system identification, the degree of 
variability of RAMP plans for key process/system identification is great. This makes any 
sort of comparative analysis between the plans difficult. While a more systemic “cut and 
paste” risk management plan process would be easier to comparatively evaluate, it would 
hold no true meaning for risk managers. The plans are specific to the contractor and 
effort at hand and rightfully so. 

3. Risk Handling Tools 

Key process owners use a pull-down menu within the RAMP database to select 

fro a listing of risk handling tools to indicate their methods of risk mitigation. If then- 

tool is not listed they may add their own. This type of selection accounts for the common 

terms used in most of the RAMP plans to indicate risk handling efforts for specific key 

processes/systems. Although many of these terms are not very descriptive in and of 
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themselves (i.e. data analysis, corrective action, product audits, etc.), their commonality 
provides some ability to comparatively analyze the chosen tools. 

“Data Analysis” was the most often applied risk handling tool (used ini084 
instances), followed by “Product Audits” (860 uses), “System Evaluation” (692 uses), 
and “Corrective Action” (468 uses). “In process” risk handling tool assignments were the 
fifth most common (329 uses) and often associated with “in process” risk rating 
assignments, but not exclusively. “Data Analysis” was a popular application across 
various Service Sets, ranking as the most popular tool in half of the One Book Chapters, 
and in the top three risk handling tools for each of the chapters 80% of the time. 

Information regarding intensity, frequency, and schedule is a data entry 
requirement for he tools and is there to provide specifics. Some plans provide verbose, 
descriptive narratives supporting their risk handling selection and application, others 
provide little at all. Similarly, some plans detail their risk handling efforts beyond the 
basic risk handling tool identification within the narrative (i.e. identify exactly what data 
items undergo “data analysis”) to add value to the risk handling selection, while others do 
not. 

So, while the system is designed to provide focal points (and create areas of 
commonality and comparability) it simultaneously provides flexibility to match the risk 
hanHling tool to the Specific problem or key process/system identified for risk 
management efforts. 

I. CHAPTER SUMMARY 

This chapter begins with an introduction of DCMDW and the sampled RAMP 
plans from this DCMA region. It provides an overview of the Overall, Service Set, and 
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One Book Chapter risk ratings for each of the 42 sampled plans. The Overall risk ratings 
and Service Set summary is presented and a brief discussion of performance, schedule, 
and cost follows. Then each of the Service Sets is presented with a summary and 
analysis of the high, moderate, and low risk ratings for each of the three areas. 
Immediately following each of the Service Sets are the applicable One Book Chapters 
and the chosen key processes/systems for rating risk for each of RAMP plans rating risk 
in this area. A summary analysis and discussion of the high risk areas follows for each of 
the One Book Chapters. Finally, variations between the sampled plans regar din g non¬ 
risk ratings, key process/system choice, and risk handling tool methodology are noted. 

The next chapter will draw conclusions regarding how DCMA addresses risk 
management in the acquisition process based on the analysis of 42 sampled RAMP plans 
within the DCMDW region. Recommendations and further areas of recommended study 
will be provided. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. OVERVIEW 

The focus of this research was to examine how DCMA addresses risk 
management in the acquisition process. To do this, the concepts of risk, risk in 
acquisition, and risk management in the context of the Federal Acquisition Process were 
presented. The role of DCMA in the post-award contract administration phase and their 
philosophy regarding risk management was discussed. DCMA’s new information 
technology tool for managing risk, RAMP was presented and a sample of 42 RAMP risk 
management plans for strategic and critical suppliers from the DCMDW region were 
studied. Performance, schedule, and cost risk ratings were examined at the Overall, 
Service Set, One Book Chapter, and key processes/system levels to determine 
commonalities and consistencies between the plans. High risk ratings and their 
associated risk handlin g tools chosen to mitigate risk were discussed in detail. Finally, 
systemic variabilities and some conclusions as to their causes were presented. 

B. CONCLUSIONS 

This research will present conclusions by answering the primary and subsidiary 
research questions proposed in Chapter 1: 

1. Subsidiary Research Question 1: What is Risk Management in the 
Context of the Federal Acquisition Process? 

The Federal Acquisition Process assesses and manages risk through all three 
phases of pre-solicitation, solicitation-award, and post-award administration. While risk 
and risk treatments may vary from phase to phase, the five-step DoD risk management 
process remains consistent throughout: risk planning, risk assessment, risk handling, risk 
monitoring, and risk docximentation. 
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Central to this is risk identification and risk analysis as part of the risk assessment 
phase. Key processes and systems are chosen for risk handling based on a measure of 
their likelihood of occurrence and their impact should the risk be realized. So probability 
and consequence are the drivers of risk assessment. Risk handling can be treated in one 
of four ways: risk avoidance, control, transfer, and assumption. 

Basic risk management in the Federal Acquisition Process is consistent with 
methods prescribed elsewhere. While there is no one formally directed way or system to 
manage risk within Government acquisition, the basic philosophies of the five-step risk 
management process and the associated risk assessment treatment and risk han dling 
options remain consistent throughout. The core foundation is the same, but the specifics 
and therefore the ability to compare risk between the various systems, agencies, and 
services is difficult at best. This is appropriate if complexity and variation of 
Government and DoD acquisition is considered, at least from the standpoint of deriving 
meaningful conclusions and not looking at wrote, summated descriptions that offer little 
scientific analysis and even less real solutions to problem solving and actual risk 
management. 

2. Subsidiary Research Question 2: What is the Defense Contract 
Management Agency (DCMA) Philosophy with regard to Risk 
Management in the Post-award Contract Administration Phase? 

DCMA is the principal contract administrator for DoD and follows the standard 
prescribed risk management process. IPTs, PROCAS, and Management Councils (all 
recognized acquisition reform initiatives) are central to DCMA’s risk management efforts 
of using a comprehensive risk management methodology inclusive of all stakeholders 
and applicable to all its suppliers. All are fully incorporated into their system for 

developing and maintaining risk management plans. 
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The nexus for risk management within DCMA is the DLAD 5000.4 One Book, in 
general, and specifically. Chapter 3.0 describing the risk management process and 
assigning responsibilities to all the CMOs. Central to this is the risk matrix structure that 
defines risk in terms of probability and consequence and assigns risk ratings in the three 
areas of performance, schedule, and cost for each of the applicable One Book Chapter 
areas and they’re associated key processes/systems. The new tool for managing this risk 
process within DCMA is RAMP, an information technology database of risk 
management plans. 

The RAMP program is a comprehensive risk management tool. It incorporates all 
five phases of the risk management process and allows for ready reference, quick update, 
and widespread information sharing within DCMA of risk management plans for various 
suppliers, programs, and contracts. Its information will be made available to customers 
(i.e. Program Managers) but it is expressly not to be used as past performance 
information. While RAMP will remain closed to suppliers, information contained within 
will be shared and discussed through other mechanisms (e.g. IPTs and Management 
Councils) to continue to promote a teaming and responsive atmosphere. 

So, the risk management philosophy within DCMA is one of comprehensive and 
inclusive management using information technology to enhance its performance and 
customer service. It’s philosophy and techniques are consistence with the Federal 
Acquisition Process and prescribed DoD methodologies. It is important to note, that 
DCMA sees this as “risk management” and not just “risk handling” or “risk monitoring” 
because its key tool, RAMP incorporates all five phases of the risk management process 
and it is iterative and timely. 
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In actuality, it is the suppliers’ job to handle the risk for their contractual efforts. 
DCMA’s role is really one of assessment, monitoring, and documentation of the 
contractor’s systems, processes, and actual risk handling techniques. But from an 
internal standpoint of managing suppliers, DCMA is handling risk by assessing and 
monitoring problematic supplier areas (key processes/systems) and supplier methods to 
manage risk. This assessment and monitoring allows the Government (either at the 
administration level or the customer level) to take lawful and meaningful contractual 
actions to seek correction, consideration, or resolution that is fair and reasonable within 
the terms of Government procurement and the contract. 

3. Subsidiary Research Question 3; Are Risk Management Plans for 
Specific Activities Consistently Developed and Applied within 
DCMA? 

The DLAD 5000.4 One Book Chapters for the various areas addressed in the 

RAMP format for risk management plans and indeed, the RAMP program itself clearly 

promotes consistent development and application of risk management plans for the 

various geographically dispersed suppliers, contracts, and programs administratively 

managed by DCMA. While the plans are not carbon copies of each other and there are 

significant variations in narration style (descriptive and fluid v. formal and brief), depth 

of justification for assigned risk ratings (100 pages v. 10), and span of risk rating areas 

(16 One Book Chapters rated v. 1) the plans are all formatted identically and use the 

same option areas for rating risk down to the One Book Chapter level. Key processes 

and system level risk ratings present much more variability given the large number of 

choices available to functional specialists/process owners from generic sources and the 

individual contracts. While the individual plans overall may tend to use a different key 

process/system combination to address risk for an applicable One Book Chapter, there is 
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a great deal of overlap between plans of the key process/system choices leading to the 
conclusion that there is really a great deal of commonality in the choices made, especially 
for the area that lend themselves for being a bit more systemic and process oriented (i.e. 
public vouchers) vice being more contractually specific (i.e. product quality). 

Risk rating methodologies, as far as they are described, appear to be consistent. 
Performance, schedule, and cost are rated and justified according to their probability and 
impact for the specific key process or system. The process/system ratings consistently 
flow upward to justify the One Book Chapter ratings. Service Set ratings, and risk ratings 
Overall. In some senses, this is a “no brainer” because the RAMP plan promotes 
systemic pyramiding of risk ratings. But the initial key processes and systems must be 
chosen and the initial risk ratings assigned and justified. Narratives, at all levels, are 
subjectively written and generally support those rating assignments that flow upward 
j&om the process/system assessments. 

4. Subsidiary Research Question 4: What are the Areas of Highest Risk 
for Strategic and Critical Suppliers in the Contract Administration 
Phase? 

Performance was generally ranked as an area of higher risk than schedule and 
cost, although schedule closely followed due to their intrinsically close relationship and 
the lowered degree of control or affect the Government can have over these areas as 
opposed to cost. From the Government’s perspective of influence, performance and 
schedule can be influenced by the amount of money the Government is willing to pay. 
But influence is not control. From the Government’s perspective of function ownership, 
performance and schedule are strictly controlled by the contractor (i.e., the Government 
isn’t the physical builder of the weapon systems it purchases) and while cost is incurred 
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based upon the contractor’s purchasing and resource use, it doesn’t necessarily correlate 
to price paid by the Government. 

The Government can protect itself from cost overruns through fixed-price type 
contracts. That means the cost area, at least from the perspective of cost to the 
Government, is controllable by the Government to a higher degree than performance and 
schedule which ultimately lies in the hands of the contractor. Of course this assumes the 
contractor will eventually delivery the purchased performance and no additional costs 
will be incurred by the Government due to delay or failure. Whether this risk is 
appropriately accounted for under the cost area for a specific contractor, contract, or 
program in question is debatable. Obviously risk in one area will drive risk in the other 
areas. But if this is carried too far, then dividing risk into the three categories of 
performance, schedule, and cost makes no sense and one risk rating will sufiBce. From 
this view, it is reasonable to conclude some isolation of these three areas is appropriate 
otherwise they are meaningless. Generally, risk associated with failure to perform or 
schedule was addressed imder the performance and schedule area and cost was treated 
separately. 

Product Support was the area of highest risk among the Service Sets. It was the 
most applied Service Set, over 83% of the 42 plans rated risk in this area. This is due in 
part to the numerous One Book Chapters associated with this area, more than twice as 
many as any other area. Supply QA - Product Quality was the riskiest One Book 
Chapter in the RAMP system for the sample; over 76% of the plans assessed risk in this 
area. Product Quality is a One Book Chapter within the Product Support Service Set and 
further supports the “most risky” designation. Just as Product Support contains the 
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majority of One Book Chapter risk areas to support increase its likelihood of risk 
management efforts. Product Quality included far more identified key processes/systems 
than any other One Book Chapter; 140 different processes/systems were identified from 
the 42 sampled plans. 

Apart from the sheer numbers aspects of available options to rationalize the high 
risk associated with Product Support in general and Product Quality specifically, this area 
intrinsically lends itself to being the most risky. Of all the areas rated, it seems to be the 
most exactly tied to the specifics of contract performance over more generic and systemic 
risk areas. They are in congruence with the high risk conclusions regarding the areas of 
performance and schedule previously mentioned. Delivery is the second most frequently 
applied Service Set (over 76%) and Schedule and Delivery Management the second most 
frequently applied One Book Chapter risk (over 69%). 

5. Subsidiary Research Question 5: What are the Most Common Tools 
used to Mitigate Risk in Key Processes and Systems? 

“Data Analysis”, “Product Audits”, “System Evaluation”, and “Corrective 
Action” were the most commonly applied risk handling tools in the sampled RAMP 
plans. While these tools are common enough for a comparative analysis across 
individual RAMP plans. Service Sets, One Book Chapters, and processes/systems they 
are too broad for great detail or depth of analysis. When the assigned tools are supported 
by accurate intensity, frequency, and schedule information as well as narratives 
describing the selection of tools, and their applicability, they become both meaningful and 
comparable. The quality of the narratives, their descriptiveness and ability to provide 
rationale to support the tool selection are paramount to RAMP’s usefulness. Without it, 
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they remain comparable at a macro level, but without depth and substance required for 
detailed use and applicability. 

The common risk handling tools highlight DCMA’s role to analyze, monitor, and 
survey and prompt the contractor when necessary. These techniques allow the 
Government to assess problems and work with the contractor to fix them using risk 
handling techniques the contractor must ultimately employ. They focus on actions before 
the final inspection, on procedural and systemic problems to achieve real change in fixing 
the root cause of production difficulty. This allows DCMA to prioritize process 
improvement opportunities and allocate resources from a risk-based perspective. 

6. Primary Research Question: How Does the Defense Contract 
Management Agency (DCMA) Address Risk Management in the 
Acquisition Process? 

DCMA uses a comprehensive, inclusive, and iterative approach to risk 
management. It follows the Government and DoD risk management premise of using a 
five-step approach to risk management and the basic idea of identifying and assessing 
key processes/systems whose risk, either through probability or potential impact, offers 
the most cause for concern from a performance, schedule, or cost perspective. It employs 
current information technology, RAMP to provide consistency, commonality, access and 
comparability to its risk management process. 

DCMA and the RAMP process for risk management naturally focus on and 
explore high risk areas given the nature of identifying and establishing key 
processes/systems and the requirement for written narratives at every level of eissessment. 
The high risk areas tend to be related to performance and schedule, product support and 
product quality, and delivery. But risk management in the post-award phase requires 


110 



DCMA’s risk management to be more akin to risk assessment and monitoring than 
actually handling the risk, because it’s really the supplier who has the direct ability to 
make change and handle risk associated with his processes and systems. However, risk 
handling does occuT from indirect means provided through risk assessment, risk 
monitoring, and contractually corrective actions consistent with procmement laws and 
the terms of the contract The contractor’s non-responsiveness is determined by risk 
management which is a focus on those things that are important vice checking 
everything. From a cost-benefit approach, this makes economic sense. 

C. RECOMMENDATIONS 

1. DoD Should Mandate a Common Risk Management Process 
throughout all DoD Organizations and Applicable to Each of the 
Services, Agencies, and Acquisition Offices 

In times of tightening defense budgets and fewer manpower resources, DoD must 
fin d more cost efficient ways to ensure quality is delivered by its contractors. It can no 
longer depend on 100% final inspection as its primary means of surveillance. By 
evaluating high risk areas, based on probability and impact, DoD acquisition 
organizations can focus their attention on the areas where they are likely to reap the most 
results from a perspective of cost, time, and manpower input—^basically a form of cost- 
benefit analysis. 

The risk management process as defined in the DSMC Risk Management Guide is 

good starting point. However, there are numerous differing and specific plans employed 

by each of the services, agencies, and individual commands. Achieving a common 

approach is best achieved by mandating a common information technology tool. This 

would forward the ideas of interoperability between the services and the application of 

consistent Government acquisition practices. Having a common methodology that is 
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directly comparable amongst all DoD organizations improves the Government’s ability to 
learn from itself, make better acquisition decisions, and present one face to contractors. 
Commonality would open individual acquisitions to more competitors who are able to 
better understand the common DoD approach and therefore better able to compete. 

As it is currently written, RAMP would not address necessary risk management 
areas prevalent in other phases of the acquisition cycle and it references DCMA specific 
directives that are not applicable elsewhere. So, RAMP is not suitable for these purposes. 
Certainly, any information technology solution would have to be greatly expanded and 
very flexible to accommodate the differing needs and requirements of the various 
acquisition activities and services. But if a common acquisition system across DoD is an 
acquisition reform goal then moving in that direction makes formulating such a program 
much more feasible and cost effective because the variances will be fewer. Of course, 
from the direct opposite position, by formulating a DoD-wide risk management plan and 
data system the acquisition arena would move closer to having a common system. 
Designing the process and mandating the specifics is the first step. The information 
technology tool should follow. 

2. Revise the RAMP Plan Format to Make them Even More Directly 
Comparable to Each Other and Incorporate a Summated Spreadsheet 
Linked to the Risk Ratings for Each Area 

Currently all Service Sets are presented even when not rated: “not applicable” 

however, only those One Book Chapters and key processes/systems actually rated are 

presented in the final risk management plan. Expand the formatting to include all One 

Book Chapters for all rated Service Sets in the final RAMP plan whether or not they are 

rated to allow for a more direct comparison of Service Sets between plans. Additionally, 

for each of the One Book Chapters that are rated prescribe some basic key 

112 



processes/system to be commonly listed on all RAMP plans whether they are rated or 
not. Any additions to these more common areas could be added beneath to provide for 
needed detail and individuality of contractual efforts. These additions along with written 
narratives justifying risk ratings provide the necessary flexibility to include meaningful 
data for specific risk management plans into a common and generic system that is 
flexible enough to allow for deviations. 

Incorporate a summated spreadsheet, similar to that presented in Appendix B, 
linked to the actual risk ratings for each of the plans at least to the One Book Chapter 
level. This would allow for quick and easy direct comparison between plans and provide 
an accurate means to perform statistical analysis and draw summary conclusions about 
how risk is managed within a given office or geographic region. 

3. Use RAMP Data for Past Performance Information 

The inf ormation currently populating the RAMP database does not fit the DoD 

definition of Past Performance Information (PPI). This seems like an incredible waste of 
time and talent and information. The data within RAMP are factual and ciirrent. It 
studies the contractor’s processes and systems in terms of performance, schedule, and 
cost, all relevant PPI issues. Either these data and the way they are collected and 
presented should be modified to conform to DoD requirements for PPI or the DoD 
requirements regarding the collection and use of PPI should be changed to accommodate 
the wealth of information gathered in RAMP. 

High risk ratings for suppliers may be a touchy issue from their perspective. But 
it is important to remember that this rating reflects risk and not necessarily performance. 
Because the focus of RAMP data is to handle-risky areas, it may not and probably does 
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not address solid performance areas for contractors. Therefore, RAMP, if used as PPI 
should only be one of several PPI sources because its information is, by definition, 
limited to problem areas and not performance as a whole. But if used as one piece of the 
past performance map for a contractor, it will provide good detail regarding how 
responsive a contractor is once risk is identified, how able a contractor is in identifying 
and handling their risk, and their propensity to work with the Government to resolve 
problem issues. Contractors would need to be granted access to RAMP data if used for 
PPI which is not a far step fi'om the Government’s desire to share data with the contractor 
prior to data upload. 

D. SUGGESTED AREAS FOR FURTHER RESEARCH 

Suggested topics for further research include: 

1. Identify and Compare Various Risk Management Models and IT 
Systems in Use in DoD 

What are the areas of convergence and divergence between the models? How can 
risk management systems be modified to provide a “fit” for all the Services, agencies, 
and acquisition offices in DoD either in one IT system or in multiple, highly 
interoperable systems? 

2. Study the RAMP Program from an IT and Process Oriented 
Perspective 

Does RAMP improve or enhance the risk management process at DCMA? Is the 
RAMP program “user fiiendly”? Are DCMA persoimel adequately trained to use 
RAMP? Does the automated RAMP process provide meaningful data to users? Is the 
RAMP process itself a faster and more efficient means of creating, updating, and using 
risk management plans? How can the RAMP processes for data input, modification, 
retrieval, and dissemination be improved? 
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3. Research Whether RAMP and Other Risk Management Activities at 
DCMA Actually Reduce Acquisition Risk 

Overtime, do the risk ratings improve? This can be studied jBrom the standpoint of 
individual RAMP plans overtime actually documenting risk reduction, reduced risk from 
the perspective of all RAMP plans written by specific geographic or in-plant offices, or 
risk reductions documented for the regions or DCMA activities as a whole. Risk 
reductions can be viewed from the standpoint of fewer area being included and monitored 
within RAMP and from the standpoint of high risk ratings reduced to moderate or low 
ratings. 
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APPENDIX A. RAMP INVENTORY 


INDEX OF SAMPLED RAMP PLANS 

plan# dcmdw office/contractor program/contract 


DCM RAYTHEON, LOS ANGELES 

1 RAYTHEON CO CSS 

2 RAYTHEON ELEC SYS (SAT) 

3 RAYTHEON ELEC SYS (SAT) 

4 RAYTHEON ELEC SYS (SAT) 

5 RAYTHEON ELEC SYS (SAT) 

6 RAYTHEON ELEC SYS (SAT) 

DCM SAN ANTONIO 

7 HUNTSVILLE AVIATION 

8 BOEING AEROSPACE SUPT CTR 

9 RAYTHEON AIR 

10 BROWN & ROOT SVCS CORPS 

11 EXXON/MOBIL 

12 LOCKHEED MARTIN HARLINGEN 

13 GRACOIND 

14 WESTINGHOUSE ELEC CORP 

15 SOUTHWEST AIRPORT SVCS 

16 DYNA-MARQ 

17 STEWART & STEVENSON 

18 D&D MACHINERY & SALES INC 

19 DAVIES RAIL & MECHANICAL 

DCM PHOENIX 

20 MOTOROLA SSG 

21 MOTOROLA SSG 

22 MCDONNEL DOUGLAS HELICOPTER SYS 

DCM DALLAS 

23 LOCKHEED MARTIN MISSILES & FC 

24 LOCKHEED MARTIN MISSILES & FC 

25 DRS INFRARED TECHNOLOGIES LP 

26 DRS INFRARED TECHNOLOGIES LPD 

27 AMORPHOUS MATERIALS 

DCMVANNUYS 

28 LinON SYSTEM INC. G&C SYS DIV 

29 SAMS AIRPACK PLUS INC 

30 AEROJET GENERAL CORP 

DCM SANTA ANA 

31 PARKER HANNIFIN CUSTOMER 

32 APPLIED MATERIAL TECH 

33 AEROJET 

34 HONEYWELL ENGINES & SYSTEMS 

DCM LOCKHEED MARTIN, SUNNYVALE 

35 UNITED TECH CORP 

36 ASSOC AEROSPACE ACT 

37 TELECHEMINU INC 

38 NORTHROP GRUMMAN 

DCM RAYTHEON, TUCSON 

39 RAYTHEON TUCSON 

40 RAYTHEON TUCSON 

41 RAYTHEON TUCSON 

42 RAYTHEON TUCSON 


EPLRS (PRIME) 

F/A18 E/R HORNET (SUPPORT) 
F15 (SUPPORT) 

F15 (PRIME) 

F18 SPARE/SUPPORT (PRIME) 
TPCM7FIREFINDER (PRIME) 


FACILITY 

KC-10CLS (PRIME) 

FACILITY 

FACILITY 

SPO600... (PRIME) 

EELV (SUPPORT), TITAN IV (FACILITY) 
C&T (PRIME) 

FACILITY 

FACILITY 

FACILITY 

FACILITY 

FACILITY 

FACILITY 


F-22 (SUPPORT), F-22 (FACILITY) 
MAVSTAR GPS (SUPPORT) 
LONGBOW APACHE (PRIME) 


ATACMS-BAT (PRIME) (FACILITY) 

HIMARS (PRIME) 

MlA2 ABRAMS UPGRADE (PRIME) (FACILITY) 
JAVELIN DDC SUBCONTRACT (SUPPORT) 
AV-8B REMANUFACTURE (PRIME) 


FACILITY 

FACILITY 

TITAN IV (FACILITY) 


FACILITY 

FACILITY 

SADARM (PRIME) (FACILITY) 

DOD (SUPPORT), F-22 (FACILITY) 


MINUTEMAN III PRP (FACILITY) 

FACILITY 

FACILITY 

TRIDENT II MISSILE (PRIME) (FACILITY) 


TOMDEP (PRIME) TOMAHAWK (FACILITY) 
AMRAAM (PRIME) 

SYSTEMS (PRIME) 

ESSM (PRIME) 
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APPENDIX B. RISK RATING OVERVIEW 


SERVICE SETS _ 

I ONE BOOK CHAPTERS 


1. MAJOR PROGRAM RiSK RATING _ 

_ I Earned Value Management _ 

Acquisition Logistics Support _ 

Z PRODUCT SUPPORT RISK RATING ~~ 

SPRD&E - Design Engineering 
SPRD&E - Systems Engineering 

Test and Evaluation _ 

Configuration and Technical Data Management 

_ {Parts Management Program _ 

Software CAS _ 

Supplier Quality Assurance - Quality System 
Supplier Quality Assurance - Product Quality 
Packaging Management Program __ 

I ___ 

3. DBUVERYRtSK RATING _ 

_ {Schedule and Delivery Management _ 

_ {Contract Safety Requirements __ 

r ” ~ 


4. BUSiNESS AND FINANOAL SYSTEMS RISK RATING 

{Contractor Estimating System Reviews 
Material Management and Accounting Systems 

_ Contract Property Management __ 

_ {Contractor Purchasing System Reviews _ 

5. PAYMENT AND FiNANOAL MANAGEMENT RiSK RATING 

{Progress Payments Based on Costs ~ 

Public Vouchers__ 

Performance Based Payments 


1 _la_ 31 ^ 

!P E S C [PP S C !EE_§^£_ 


is |C |/P|P IS |C \iP\P js 


1 MAJOR PROGRAM RISK RATING _ 

{ Earned Value Management __ 

_ {Acquisition Logistics Support _ 

I _ 

Z PRODUCT SUPPORT RISK RATING 

_ SPRD&E - Design Engineering _ 

ISPRD&E - Systems Engineering 

Test and Evaluation _ 

Confi guration and Technical Data Management 

_ {Parts Management Program __ 

_ {Software CAS ___ 

{Supplier Quality Assurance - Quality System 
{Supplier Quality Assurance - Product Quality 
_ {Packaging Management Program _ 

r ' ” 


3. DBUVERYRtSK RATING _ 

{Schedule and Delivery Management 

Contract Safety Requirements _ 

4. BUSINESS AND RNANaAL SYSTEMS RiSK RATING 

Contractor Estimating System Reviews 
Material Management and Accounting Systems 

{Contract Property Management _ 

_ {Contractor Purchasing System Reviews _ 

5. PAYMENT AND HNANaAL MANAGEMENT RISK RAVNG~ 

{Progress Payments Based on Costs _ 

Public Vouchers _ 

” Performance Based Payments 


aaaMtaaBEaEjaBEHiaagggBgaaa 

BBBBBBBBBBBBBBBBBBBBaaa 
BBBBBBBBBBBBBBBBBBBBBBB 
BBBBBBBBBBBBBBBBBBBBBBB 

Jbbbbbbbbbbbbbbbbbbbbbbb 

iBaaaBaEaaBBBBBBaaBaaaBaBB 

—^■■■■bbBBBBBBBBBBBBBBB 

Jbbbbbbbbbbbbbbbbbbbb 

iBBBBBBBBBBBBBBBBBBBBBBBB 

BBBBBBBBBBBBBBBBBBBBBBBB 

'---JBBBBBBBBBBBBBBBBBBBB 
iBBBBBBBBBBBBBBBBBBBBBBBB 
-JBBBBBBBBBBBBBBBBBBBB 

BGDIBBIBIIIBBBISIBBBBBIZIBIZIBB 


_icacaBBaaBBaBa 

IBBBBBBBBBBBBBaaaBaaBBBBB 

■BBBBBBBBBBBBBBBBBBBBBBB 
■BBBBBBBBBBBBBBBBBBBBBBB 
laaaBBBBBBBBBBBBBBBBBBBB 
■BBBBBBBBBBBBBBBBBBBBBBB 

jaaga aaagaaaa aaa 
qaaa a8aa8888B8888S88 88 

!38888a88888 3aa8 8aaa a 

iMHH. 


IP\P \s c 


SSSaaaaBaaa aaaaBaa a 

BaBBaBBBBaBBBBBBBBr 
BBBBBBBBaaaBBBBBBBI 
BBBBBBBBaaaBBBBBBBI 
BBBBBBBBBBBBBBBBBBI 

__■BBBBBBBBBBBBBBBBBBI, 

OBBBaaaBBBBBBBBBaaaaBBB 


BBBBBBBBBBBB^ 

BBBBaaaBBBBB^ 


CiaE9BC3CIia^BC3E9E9^ 

BBBBaaaBBBB^ 

BBBBBBBBBBB^ 







_IBBBBBBBBBBBaaaBBBBBBBI , 

BBBBaaaBBBBBBBBBBBBBBBB 


119 














120 











































pggggggggK 

■SSaHiacicaBaaia 



1 major program RISK RATING _ 

Earned Value Management 

Acquisition Logistics Support _ 

Z PRODUCT SUPPORT RISK RATING _ 


iSPRD&E - Design Engineering_ 


SPRD&E - Systems Engineenng _ 

Test and Evaluation ___ 

Co nfiguration and Technical Data Management 

Parts Management Program _ 

Software CAS ___ 

Supplier Quality Assurance - Quality System 
~ Supplier Quality Assurance - Product Quality ~ 
Packaging Management Program_ 
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[Schedule and Delivery Management 
[contract Safety Requirements 


4. BUSINESS AND FINANCIAL SYSTEMS RISK RATING 
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